Jul 07 10:25:28.588596 ip-10-0-139-204 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 10:25:28.588607 ip-10-0-139-204 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 10:25:28.588616 ip-10-0-139-204 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 10:25:28.588936 ip-10-0-139-204 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 10:25:38.713922 ip-10-0-139-204 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 10:25:38.713938 ip-10-0-139-204 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 09dceb71b9dc42efbe706017e35e553f -- Jul 07 10:27:11.353562 ip-10-0-139-204 systemd[1]: Starting Kubernetes Kubelet... Jul 07 10:27:11.776771 ip-10-0-139-204 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 10:27:11.776771 ip-10-0-139-204 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 10:27:11.776771 ip-10-0-139-204 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 10:27:11.776771 ip-10-0-139-204 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 10:27:11.776771 ip-10-0-139-204 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 10:27:11.777941 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.777854 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 10:27:11.781695 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781680 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781698 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781703 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781706 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781709 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781712 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781714 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781718 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781721 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781724 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781726 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781729 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781731 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781734 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781736 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781739 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781741 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781745 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.781738 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781747 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781756 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781759 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781762 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781764 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781766 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781770 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781774 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781777 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781779 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781782 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781784 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781787 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781789 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781792 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781794 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781796 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781799 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781801 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781804 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.782163 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781806 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781808 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781811 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781813 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781815 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781818 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781820 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781823 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781825 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781830 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781832 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781836 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781838 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781840 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781843 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781846 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781848 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781851 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781853 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781856 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.782703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781858 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781860 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781863 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781865 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781867 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781870 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781872 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781875 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781877 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781880 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781882 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781884 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781887 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781889 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781891 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781894 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781898 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781900 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781902 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781905 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.783180 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781907 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781910 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781913 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781916 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781918 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781920 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781924 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781926 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781929 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781931 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781934 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781936 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781939 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781941 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781944 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781946 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781949 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781951 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781953 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781955 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.783679 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.781958 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783899 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783907 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783911 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783913 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783916 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783919 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783922 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783925 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783929 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783932 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783935 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783938 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783940 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783943 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783946 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783949 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783952 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783954 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783956 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.784152 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783959 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783961 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783963 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783966 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783968 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783971 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783973 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783975 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783978 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783980 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783983 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783985 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783987 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783990 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783993 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783996 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.783999 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784001 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784005 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.784670 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784008 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784011 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784014 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784017 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784019 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784022 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784025 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784028 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784030 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784033 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784037 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784041 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784043 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784046 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784049 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784051 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784054 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784056 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784059 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784061 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.785120 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784064 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784067 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784070 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784072 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784075 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784077 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784080 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784083 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784085 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784088 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784090 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784093 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784095 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784098 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784100 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784102 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784105 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784107 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784110 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.785668 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784112 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784115 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784117 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784120 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784122 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784125 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784127 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784130 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784132 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784134 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784136 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784139 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784141 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784144 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784146 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784148 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784151 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784153 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784155 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784158 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.786138 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784161 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784164 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784561 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784566 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784570 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784572 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784575 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784578 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784580 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784583 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784586 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784588 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784591 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784593 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784596 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784598 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784601 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784604 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784607 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.786650 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784610 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784612 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784614 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784617 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784619 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784621 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784624 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784626 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784629 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784632 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784634 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784637 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784639 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784643 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784646 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784648 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784651 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784653 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784656 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784658 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.787113 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784661 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784664 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784666 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784668 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784671 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784674 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784676 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784679 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784681 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784687 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784690 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784693 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784695 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784697 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784700 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784702 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784705 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784707 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784710 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784712 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.787626 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784714 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784717 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784720 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784722 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784725 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784728 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784731 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784733 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784735 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784738 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784740 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784743 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784746 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784748 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784750 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784753 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784755 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784757 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784760 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784762 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.788101 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784765 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784767 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784770 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784774 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784777 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784780 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784797 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784801 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784804 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784807 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784809 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784814 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784816 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784819 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784822 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784825 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784829 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784832 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784836 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.788592 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784839 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784842 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.784844 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784913 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784920 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784926 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784931 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784935 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784938 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784943 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784947 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784950 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784953 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784956 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784959 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784962 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784969 2581 flags.go:64] FLAG: --cgroup-root="" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784972 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784975 2581 flags.go:64] FLAG: --client-ca-file="" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784978 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784981 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784985 2581 flags.go:64] FLAG: --cluster-domain="" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784988 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784990 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 10:27:11.789042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784995 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.784998 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785001 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785004 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785007 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785010 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785014 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785017 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785019 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785022 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785025 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785029 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785032 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785035 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785038 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785040 2581 flags.go:64] FLAG: --enable-server="true" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785044 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785050 2581 flags.go:64] FLAG: --event-burst="100" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785053 2581 flags.go:64] FLAG: --event-qps="50" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785056 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785059 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785062 2581 flags.go:64] FLAG: --eviction-hard="" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785066 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785068 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785071 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 10:27:11.789646 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785080 2581 flags.go:64] FLAG: --eviction-soft="" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785083 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785086 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785089 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785091 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785094 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785097 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785101 2581 flags.go:64] FLAG: --feature-gates="" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785105 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785108 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785111 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785115 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785118 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785122 2581 flags.go:64] FLAG: --help="false" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785125 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-139-204.ec2.internal" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785128 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785131 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785134 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785137 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785140 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785143 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785146 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785149 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 10:27:11.790224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785152 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785154 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785157 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785160 2581 flags.go:64] FLAG: --kube-reserved="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785163 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785165 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785168 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785171 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785174 2581 flags.go:64] FLAG: --lock-file="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785176 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785185 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785188 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785193 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785196 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785198 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785201 2581 flags.go:64] FLAG: --logging-format="text" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785206 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785209 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785212 2581 flags.go:64] FLAG: --manifest-url="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785214 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785219 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785221 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785228 2581 flags.go:64] FLAG: --max-pods="110" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785231 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785234 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 10:27:11.790848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785237 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785240 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785243 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785246 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785248 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785255 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785258 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785275 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785278 2581 flags.go:64] FLAG: --pod-cidr="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785281 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785287 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785290 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785293 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785295 2581 flags.go:64] FLAG: --port="10250" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785298 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785301 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-057ece6c9471691bf" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785304 2581 flags.go:64] FLAG: --qos-reserved="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785307 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785310 2581 flags.go:64] FLAG: --register-node="true" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785313 2581 flags.go:64] FLAG: --register-with-taints="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785317 2581 flags.go:64] FLAG: --registry-burst="10" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785319 2581 flags.go:64] FLAG: --registry-qps="5" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785322 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785325 2581 flags.go:64] FLAG: --reserved-memory="" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785330 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 10:27:11.791488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785333 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785336 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785339 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785342 2581 flags.go:64] FLAG: --runonce="false" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785346 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785349 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785352 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785354 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785357 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785360 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785363 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785366 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785369 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785372 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785375 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785377 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785380 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785383 2581 flags.go:64] FLAG: --system-cgroups="" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785386 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785391 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785393 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785396 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785402 2581 flags.go:64] FLAG: --tls-min-version="" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785405 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785408 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 10:27:11.792101 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785410 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785414 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785417 2581 flags.go:64] FLAG: --v="2" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785420 2581 flags.go:64] FLAG: --version="false" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785424 2581 flags.go:64] FLAG: --vmodule="" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785427 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.785432 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785544 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785548 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785551 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785553 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785557 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785560 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785562 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785565 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785567 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785570 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785572 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785575 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785577 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785579 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785582 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.792723 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785584 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785587 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785589 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785592 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785594 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785597 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785599 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785601 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785604 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785607 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785609 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785612 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785615 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785617 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785620 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785622 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785626 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785628 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785631 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785633 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.793254 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785636 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785640 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785642 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785644 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785647 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785650 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785653 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785655 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785658 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785660 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785663 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785665 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785667 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785670 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785672 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785675 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785678 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785680 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785683 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785685 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.793755 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785688 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785690 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785693 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785695 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785697 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785700 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785703 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785705 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785709 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785711 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785713 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785716 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785718 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785722 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785726 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785729 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785732 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785735 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785737 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.794285 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785739 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785742 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785744 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785747 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785750 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785752 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785754 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785757 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785759 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785763 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785766 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785769 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785771 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785773 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785776 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785778 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785781 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785783 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785785 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.794747 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785789 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785793 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785797 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785800 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785802 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.785805 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.786675 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.793786 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.793800 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.793822 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.793828 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793887 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793892 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793895 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793898 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.795202 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793901 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793904 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793907 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793909 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793912 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793914 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793917 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793920 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793922 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793925 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793927 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793930 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793933 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793935 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793938 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793941 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793943 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793946 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793949 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793951 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.795605 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793954 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793956 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793959 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793963 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793967 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793970 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793973 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793975 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793978 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793980 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793983 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793986 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793988 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793991 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793993 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793996 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.793998 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794001 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794003 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794005 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.796081 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794008 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794010 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794013 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794015 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794017 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794020 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794022 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794025 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794027 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794030 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794032 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794035 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794037 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794040 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794042 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794045 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794047 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794051 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794054 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.796582 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794056 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794059 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794061 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794064 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794066 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794069 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794073 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794076 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794079 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794083 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794086 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794089 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794092 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794095 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794097 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794100 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794102 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794104 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794107 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.797032 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794109 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794112 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794115 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794117 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794119 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794123 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794125 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794127 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794131 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794133 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794135 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794138 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794141 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794143 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794146 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794148 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.797504 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794151 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.794156 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794301 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794306 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794310 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794312 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794315 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794318 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794321 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794323 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794326 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794329 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794331 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794334 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794337 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 10:27:11.797950 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794339 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794342 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794344 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794346 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794349 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794351 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794353 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794356 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794358 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794361 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794363 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794365 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794369 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794371 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794373 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794376 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794378 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794381 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794384 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 10:27:11.798441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794386 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794389 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794391 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794394 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794396 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794399 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794401 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794403 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794406 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794408 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794410 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794413 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794416 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794418 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794420 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794424 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794427 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794430 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794432 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 10:27:11.798888 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794435 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794438 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794441 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794443 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794446 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794448 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794451 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794455 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794458 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794460 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794463 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794465 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794468 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794470 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794472 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794475 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794477 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794479 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794482 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 10:27:11.799359 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794484 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794486 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794489 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794491 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794494 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794496 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794498 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794501 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794503 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794506 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794508 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794510 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794512 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794515 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794517 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794520 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794523 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794525 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794527 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794530 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 10:27:11.799804 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794533 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794535 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794537 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794540 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794542 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794545 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794549 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794552 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:11.794555 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.794559 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.794668 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.798009 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.799253 2581 server.go:1019] "Starting client certificate rotation" Jul 07 10:27:11.800291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.799363 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 10:27:11.800620 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.799398 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 10:27:11.801683 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.801670 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 10:27:11.803224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.803207 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 10:27:11.813951 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.813927 2581 log.go:25] "Validated CRI v1 runtime API" Jul 07 10:27:11.816244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.816225 2581 log.go:25] "Validated CRI v1 image API" Jul 07 10:27:11.817011 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.816998 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 10:27:11.819620 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.819600 2581 fs.go:135] Filesystem UUIDs: map[2595651f-7b08-4a6e-ba7c-fdbbfcc407eb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 96e1ca4d-f7ac-4a0d-8544-c30c17fb429f:/dev/nvme0n1p3] Jul 07 10:27:11.819690 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.819619 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 10:27:11.824918 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.824811 2581 manager.go:216] Machine: {Timestamp:2026-07-07 10:27:11.824296026 +0000 UTC m=+0.346364530 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102673 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23fdc43c7017e513de6489190bcaab SystemUUID:ec23fdc4-3c70-17e5-13de-6489190bcaab BootID:09dceb71-b9dc-42ef-be70-6017e35e553f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:7c:c2:4a:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:7c:c2:4a:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:c7:c7:2c:90:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 10:27:11.824918 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.824915 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 10:27:11.825025 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.824989 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 10:27:11.830576 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.830547 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 10:27:11.830709 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.830579 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 10:27:11.831273 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.831250 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 10:27:11.831308 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.831278 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 10:27:11.831308 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.831286 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 10:27:11.832538 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.832525 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 10:27:11.833732 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.833722 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 10:27:11.834934 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.834919 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 10:27:11.837601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837589 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 07 10:27:11.837713 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837704 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 10:27:11.837756 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837734 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 07 10:27:11.837756 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837744 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 10:27:11.837824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837758 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 10:27:11.837824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837767 2581 kubelet.go:398] "Adding apiserver pod source" Jul 07 10:27:11.837824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.837777 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 10:27:11.838155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.838143 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 10:27:11.838189 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.838163 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 10:27:11.840237 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.840217 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 10:27:11.842123 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.842110 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 10:27:11.842180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.842140 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 10:27:11.843232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843211 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 10:27:11.843232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843232 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843239 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843245 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843254 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843275 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843286 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843292 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843300 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843305 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843322 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 10:27:11.843388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.843328 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 10:27:11.844107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.844096 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 10:27:11.844107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.844106 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 10:27:11.846856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.846825 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 10:27:11.847145 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.847127 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 10:27:11.847198 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.847126 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 10:27:11.847859 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.847846 2581 server.go:1268] "Started kubelet" Jul 07 10:27:11.847983 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.847951 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 10:27:11.848123 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.847994 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 10:27:11.848216 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.848205 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 10:27:11.850117 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.850089 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 10:27:11.850300 ip-10-0-139-204 systemd[1]: Started Kubernetes Kubelet. Jul 07 10:27:11.852923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.852906 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 07 10:27:11.853750 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.852883 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-204.ec2.internal.18bffb11eb9ae349 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-204.ec2.internal,UID:ip-10-0-139-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-204.ec2.internal,},FirstTimestamp:2026-07-07 10:27:11.847818057 +0000 UTC m=+0.369886543,LastTimestamp:2026-07-07 10:27:11.847818057 +0000 UTC m=+0.369886543,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-204.ec2.internal,}" Jul 07 10:27:11.854983 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.854963 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 10:27:11.856917 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.856900 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 10:27:11.857007 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.856904 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 10:27:11.857059 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.857040 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:11.858816 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.858746 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 10:27:11.858897 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.858844 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 10:27:11.858897 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.858861 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 10:27:11.859338 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.859323 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 07 10:27:11.859338 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.859337 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 07 10:27:11.861920 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.861904 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 10:27:11.861920 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.861919 2581 factory.go:55] Registering systemd factory Jul 07 10:27:11.861920 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.861926 2581 factory.go:223] Registration of the systemd container factory successfully Jul 07 10:27:11.862146 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.862127 2581 factory.go:153] Registering CRI-O factory Jul 07 10:27:11.862146 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.862143 2581 factory.go:223] Registration of the crio container factory successfully Jul 07 10:27:11.862246 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.862161 2581 factory.go:103] Registering Raw factory Jul 07 10:27:11.862246 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.862173 2581 manager.go:1196] Started watching for new ooms in manager Jul 07 10:27:11.862665 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.862624 2581 manager.go:349] Starting recovery of all containers Jul 07 10:27:11.863328 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.863294 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 10:27:11.863399 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.863368 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 10:27:11.872437 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.872309 2581 manager.go:354] Recovery completed Jul 07 10:27:11.873703 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.873683 2581 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 10:27:11.877025 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.877013 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:11.880349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880317 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:11.880349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880343 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4swq" Jul 07 10:27:11.880443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880363 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:11.880443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880376 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:11.880946 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880923 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 10:27:11.880946 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880937 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 10:27:11.881039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880951 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:11.881039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.880954 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 07 10:27:11.881689 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.881677 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:11.881729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.881704 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:11.881729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.881717 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:11.882290 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.882278 2581 policy_none.go:49] "None policy: Start" Jul 07 10:27:11.882337 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.882294 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 10:27:11.882337 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.882305 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 10:27:11.882902 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.882817 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-204.ec2.internal.18bffb11ed8b4999 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-204.ec2.internal,UID:ip-10-0-139-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-204.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-204.ec2.internal,},FirstTimestamp:2026-07-07 10:27:11.880350105 +0000 UTC m=+0.402418590,LastTimestamp:2026-07-07 10:27:11.880350105 +0000 UTC m=+0.402418590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-204.ec2.internal,}" Jul 07 10:27:11.883328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.883316 2581 policy_none.go:47] "Start" Jul 07 10:27:11.893577 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.893503 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-204.ec2.internal.18bffb11ed8b9197 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-204.ec2.internal,UID:ip-10-0-139-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-204.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-204.ec2.internal,},FirstTimestamp:2026-07-07 10:27:11.880368535 +0000 UTC m=+0.402437020,LastTimestamp:2026-07-07 10:27:11.880368535 +0000 UTC m=+0.402437020,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-204.ec2.internal,}" Jul 07 10:27:11.898989 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.898968 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r4swq" Jul 07 10:27:11.902200 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.902120 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-204.ec2.internal.18bffb11ed8bbd65 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-204.ec2.internal,UID:ip-10-0-139-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-204.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-204.ec2.internal,},FirstTimestamp:2026-07-07 10:27:11.880379749 +0000 UTC m=+0.402448234,LastTimestamp:2026-07-07 10:27:11.880379749 +0000 UTC m=+0.402448234,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-204.ec2.internal,}" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.919866 2581 manager.go:337] "Starting Device Plugin manager" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.919900 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.919910 2581 server.go:87] "Starting device plugin registration server" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.920146 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.920159 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.920418 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.920501 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:11.920509 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.921009 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 10:27:11.934608 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:11.921058 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.009790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.009742 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 10:27:12.011106 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.011091 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 10:27:12.011168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.011126 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 10:27:12.011168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.011149 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 10:27:12.011233 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.011180 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 10:27:12.013651 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.013634 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 10:27:12.020726 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.020712 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.021702 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.021679 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.021796 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.021711 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.021796 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.021724 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.021796 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.021747 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.032482 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.032430 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.032482 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.032452 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-204.ec2.internal\": node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.055053 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.055024 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.112417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.111794 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal"] Jul 07 10:27:12.112538 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.112465 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.113396 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.113383 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.113443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.113413 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.113443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.113422 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.114653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.114640 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.114791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.114775 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.114834 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.114810 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.115375 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115359 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.115375 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115370 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.115466 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115391 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.115466 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115391 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.115466 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115406 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.115466 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.115414 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.116450 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.116431 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.116560 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.116459 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.117069 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.117054 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.117142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.117080 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.117142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.117093 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.132823 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.132802 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-204.ec2.internal\" not found" node="ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.136661 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.136645 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-204.ec2.internal\" not found" node="ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.155281 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.155240 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.160482 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.160462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.160561 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.160496 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.160561 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.160524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9585bff18ee532ee27e6d3ba6f76cb3f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-204.ec2.internal\" (UID: \"9585bff18ee532ee27e6d3ba6f76cb3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.256244 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.256214 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.260626 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.260693 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.260693 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9585bff18ee532ee27e6d3ba6f76cb3f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-204.ec2.internal\" (UID: \"9585bff18ee532ee27e6d3ba6f76cb3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.260782 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260697 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9585bff18ee532ee27e6d3ba6f76cb3f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-204.ec2.internal\" (UID: \"9585bff18ee532ee27e6d3ba6f76cb3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.260782 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.260782 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.260724 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/81b239ad80af5a4b339428b7893ca0f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal\" (UID: \"81b239ad80af5a4b339428b7893ca0f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.357059 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.356994 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.437520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.437485 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.439649 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.439633 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:12.457292 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.457254 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.557788 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.557748 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.658303 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.658213 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.758749 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.758711 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.799119 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.799095 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 10:27:12.799700 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.799232 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 10:27:12.845192 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.845160 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 10:27:12.859707 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.859680 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:12.901038 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.900986 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 10:22:12 +0000 UTC" deadline="2028-03-31 12:21:29.941997052 +0000 UTC" Jul 07 10:27:12.901038 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.901035 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15193h54m17.040965327s" Jul 07 10:27:12.927305 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.927279 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 10:27:12.928643 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.928620 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientMemory" Jul 07 10:27:12.928766 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.928663 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 10:27:12.928766 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:12.928679 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeHasSufficientPID" Jul 07 10:27:12.960549 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:12.960524 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:13.016074 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:13.016039 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b239ad80af5a4b339428b7893ca0f3.slice/crio-c0e27f9fa59cc5b597f97671d59893d9dbb329a8ce71ff61812d205e023e2d43 WatchSource:0}: Error finding container c0e27f9fa59cc5b597f97671d59893d9dbb329a8ce71ff61812d205e023e2d43: Status 404 returned error can't find the container with id c0e27f9fa59cc5b597f97671d59893d9dbb329a8ce71ff61812d205e023e2d43 Jul 07 10:27:13.016416 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:13.016392 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9585bff18ee532ee27e6d3ba6f76cb3f.slice/crio-6ac784f1f02308a44e69a0ee8ba78aded836908bb9e3e466b85c6e9fecfadfa7 WatchSource:0}: Error finding container 6ac784f1f02308a44e69a0ee8ba78aded836908bb9e3e466b85c6e9fecfadfa7: Status 404 returned error can't find the container with id 6ac784f1f02308a44e69a0ee8ba78aded836908bb9e3e466b85c6e9fecfadfa7 Jul 07 10:27:13.021198 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:13.021181 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 10:27:13.046380 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:13.046358 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 10:27:13.060908 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:13.060886 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:13.161472 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:13.161443 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:14.323597 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:14.323557 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-204.ec2.internal\" not found" Jul 07 10:27:14.382560 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.382527 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 10:27:14.418967 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.418935 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" Jul 07 10:27:14.432168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.432138 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 10:27:14.433164 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.433148 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" Jul 07 10:27:14.441682 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.441657 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 10:27:14.899727 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.899691 2581 apiserver.go:52] "Watching apiserver" Jul 07 10:27:14.911753 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.911719 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 10:27:14.912243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.912211 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-wr9zd","openshift-image-registry/node-ca-vfdgc","openshift-multus/multus-25wmd","openshift-multus/network-metrics-daemon-hgc7q","openshift-network-operator/iptables-alerter-ss4ml","openshift-ovn-kubernetes/ovnkube-node-76p9p","kube-system/global-pull-secret-syncer-h9wvp","kube-system/konnectivity-agent-p7s6m","kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal","openshift-multus/multus-additional-cni-plugins-6xhsq","openshift-network-diagnostics/network-check-target-sdbtk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29","openshift-cluster-node-tuning-operator/tuned-5nhb7"] Jul 07 10:27:14.913937 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.913915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:14.914052 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:14.913998 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:14.915097 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.915075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.916246 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.916225 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.917273 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.917186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:14.917273 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:14.917239 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:14.918352 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.918329 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 10:27:14.918642 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.918628 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.919010 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.918985 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.919139 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919121 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jd76f\"" Jul 07 10:27:14.919184 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919158 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 10:27:14.919364 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919345 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 10:27:14.919364 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919361 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.919552 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919538 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 10:27:14.919855 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919834 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.920036 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.920013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.920920 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.919791 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:14.921165 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.921145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.923902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.922976 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-27pvk\"" Jul 07 10:27:14.923902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.923286 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 10:27:14.925841 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.924413 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.925841 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.924929 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.925841 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.925825 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 10:27:14.926135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926085 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926286 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926431 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926514 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926535 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926645 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rkq9g\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926786 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 10:27:14.926902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926883 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:14.927270 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926918 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6xbbt\"" Jul 07 10:27:14.927270 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.926993 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.927270 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.927077 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.927457 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.927436 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8ckfm\"" Jul 07 10:27:14.928457 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.928441 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:14.929538 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.929522 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:14.930658 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.930639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:14.930751 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:14.930705 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:14.930957 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.930931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-multus-certs\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931041 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.930969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-sys\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.931041 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.930994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-socket-dir-parent\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931041 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931018 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmfr8\" (UniqueName: \"kubernetes.io/projected/cdb94fad-2c12-4d5b-94ee-c31a4f061916-kube-api-access-vmfr8\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.931155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:14.931155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd9495b-b6b5-43a1-891c-d409dffab963-host-slash\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:14.931155 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-systemd-units\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-netd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-script-lib\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-netns\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931251 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-os-release\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931274 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-bin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-conf-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-systemd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-config\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931456 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kw7f\" (UniqueName: \"kubernetes.io/projected/a240c555-49f6-47cb-9a8e-7f43b03d2985-kube-api-access-9kw7f\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cni-binary-copy\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931492 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-lib-modules\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931507 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/601895bc-5e06-4dd8-bdf1-43babbb67529-hosts-file\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-hostroot\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxvjr\" (UniqueName: \"kubernetes.io/projected/a6cd8d74-720e-4c12-b299-21b412df7af2-kube-api-access-vxvjr\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbtfn\" (UniqueName: \"kubernetes.io/projected/e3e80296-decf-49a0-a5c9-71630ad65571-kube-api-access-bbtfn\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-tmp\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.931668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931650 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn575\" (UniqueName: \"kubernetes.io/projected/601895bc-5e06-4dd8-bdf1-43babbb67529-kube-api-access-pn575\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-netns\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931728 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.931752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-sys-fs\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-var-lib-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-etc-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-etc-kubernetes\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-ovn\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932269 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-systemd\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.932317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/601895bc-5e06-4dd8-bdf1-43babbb67529-tmp-dir\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-k8s-cni-cncf-io\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-daemon-config\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-kubernetes\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932431 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-device-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932456 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-etc-selinux\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxp6w\" (UniqueName: \"kubernetes.io/projected/dcd9495b-b6b5-43a1-891c-d409dffab963-kube-api-access-rxp6w\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-slash\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-modprobe-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932556 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysconfig\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-host\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932602 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-dbus\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932625 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-socket-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-kubelet\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-node-log\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.932774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932695 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-bin\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-env-overrides\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6cd8d74-720e-4c12-b299-21b412df7af2-ovn-node-metrics-cert\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932783 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-var-lib-kubelet\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.932826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-kubelet-config\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933196 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfplp\" (UniqueName: \"kubernetes.io/projected/898634ce-48a8-473d-9809-2ad41c4fa11c-kube-api-access-qfplp\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-run\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-tuned\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-multus\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933339 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-kubelet\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933391 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-registration-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd9495b-b6b5-43a1-891c-d409dffab963-iptables-alerter-script\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-log-socket\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-conf\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-system-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.933537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.933505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cnibin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:14.934739 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.934715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 10:27:14.934848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.934774 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.934997 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vb56t\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935284 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935322 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935521 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935578 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935287 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935744 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jjvj8\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935938 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hbjnp\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.935521 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.936099 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cq8kd\"" Jul 07 10:27:14.936515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.936258 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 10:27:14.938228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.938202 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 10:27:14.956982 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.956955 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r7h5g" Jul 07 10:27:14.965292 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:14.965271 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r7h5g" Jul 07 10:27:15.021317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.021293 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 10:27:15.034663 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxp6w\" (UniqueName: \"kubernetes.io/projected/dcd9495b-b6b5-43a1-891c-d409dffab963-kube-api-access-rxp6w\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-slash\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034697 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-modprobe-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034721 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysconfig\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-host\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034769 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-dbus\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.034843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-socket-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-kubelet\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-node-log\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-bin\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-env-overrides\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6cd8d74-720e-4c12-b299-21b412df7af2-ovn-node-metrics-cert\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.034972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-var-lib-kubelet\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035050 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-node-log\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035401 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-kubelet-config\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.035401 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfplp\" (UniqueName: \"kubernetes.io/projected/898634ce-48a8-473d-9809-2ad41c4fa11c-kube-api-access-qfplp\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.035401 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035351 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 10:27:15.035526 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035413 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-run\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035526 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035446 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-slash\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035536 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-modprobe-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035574 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-var-lib-kubelet\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysconfig\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035664 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-kubelet\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035696 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-dbus\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.035866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035734 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2087d4e-9d80-4d94-b537-077d2901c6fa-kubelet-config\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.035866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-run\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.035866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035840 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-bin\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.035866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035848 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-tuned\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.036064 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035892 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-socket-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.036064 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.035981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-multus\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036064 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-kubelet\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036080 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-host\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bcc27c-62e9-4172-8855-09ac0c8cb194-host\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036114 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-kubelet\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-registration-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd9495b-b6b5-43a1-891c-d409dffab963-iptables-alerter-script\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.036186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036176 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-registration-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-log-socket\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036220 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-multus\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036235 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-conf\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036279 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-system-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cnibin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036319 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-env-overrides\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-multus-certs\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036371 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-conf\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036381 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cnibin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036399 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-log-socket\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-sys\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036445 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-system-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.036469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-socket-dir-parent\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmfr8\" (UniqueName: \"kubernetes.io/projected/cdb94fad-2c12-4d5b-94ee-c31a4f061916-kube-api-access-vmfr8\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-socket-dir-parent\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036497 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-multus-certs\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-os-release\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036537 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-sys\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036766 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-sysctl-d\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036765 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd9495b-b6b5-43a1-891c-d409dffab963-host-slash\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036841 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-systemd-units\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036820 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd9495b-b6b5-43a1-891c-d409dffab963-iptables-alerter-script\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036863 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd9495b-b6b5-43a1-891c-d409dffab963-host-slash\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.036864 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-netd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036912 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-systemd-units\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.036958 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-script-lib\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.037005 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:15.536965177 +0000 UTC m=+2.997499917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55bcc27c-62e9-4172-8855-09ac0c8cb194-serviceca\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-cni-netd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037108 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92bdf557-5f55-49e0-9c91-eb18897f1e92-konnectivity-ca\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037166 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-netns\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037207 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-os-release\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-bin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037235 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-netns\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-conf-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-systemd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037284 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-var-lib-cni-bin\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037254 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-cni-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037303 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-os-release\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.037958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037322 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-config\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037333 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-conf-dir\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kw7f\" (UniqueName: \"kubernetes.io/projected/a240c555-49f6-47cb-9a8e-7f43b03d2985-kube-api-access-9kw7f\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037366 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-systemd\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037373 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037375 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cni-binary-copy\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-lib-modules\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/601895bc-5e06-4dd8-bdf1-43babbb67529-hosts-file\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-hostroot\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxvjr\" (UniqueName: \"kubernetes.io/projected/a6cd8d74-720e-4c12-b299-21b412df7af2-kube-api-access-vxvjr\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbtfn\" (UniqueName: \"kubernetes.io/projected/e3e80296-decf-49a0-a5c9-71630ad65571-kube-api-access-bbtfn\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037513 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-script-lib\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92bdf557-5f55-49e0-9c91-eb18897f1e92-agent-certs\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037574 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-tmp\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn575\" (UniqueName: \"kubernetes.io/projected/601895bc-5e06-4dd8-bdf1-43babbb67529-kube-api-access-pn575\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-netns\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.038752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037658 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037704 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-sys-fs\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6cd8d74-720e-4c12-b299-21b412df7af2-ovnkube-config\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-var-lib-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-etc-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-etc-kubernetes\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037843 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/601895bc-5e06-4dd8-bdf1-43babbb67529-hosts-file\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-system-cni-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-ovn\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037898 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-cni-binary-copy\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-systemd\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037947 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-lib-modules\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037954 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-hostroot\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037999 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-netns\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-sys-fs\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.037951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/601895bc-5e06-4dd8-bdf1-43babbb67529-tmp-dir\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.039498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038193 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-k8s-cni-cncf-io\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038199 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/601895bc-5e06-4dd8-bdf1-43babbb67529-tmp-dir\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038214 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-etc-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-daemon-config\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038236 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-run-ovn\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038267 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-systemd\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-host-run-ovn-kubernetes\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038294 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cnibin\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038317 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmnzp\" (UniqueName: \"kubernetes.io/projected/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-kube-api-access-xmnzp\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038363 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038376 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6cd8d74-720e-4c12-b299-21b412df7af2-var-lib-openvswitch\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-kubernetes\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-host-run-k8s-cni-cncf-io\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-799w7\" (UniqueName: \"kubernetes.io/projected/55bcc27c-62e9-4172-8855-09ac0c8cb194-kube-api-access-799w7\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.038479 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038531 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-kubernetes\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038536 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cdb94fad-2c12-4d5b-94ee-c31a4f061916-etc-kubernetes\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.040349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-device-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.038580 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:15.538562905 +0000 UTC m=+2.999097635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-etc-selinux\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038624 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-device-dir\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038640 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cdb94fad-2c12-4d5b-94ee-c31a4f061916-multus-daemon-config\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.038728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/898634ce-48a8-473d-9809-2ad41c4fa11c-etc-selinux\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.039369 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-etc-tuned\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.039990 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6cd8d74-720e-4c12-b299-21b412df7af2-ovn-node-metrics-cert\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.041079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.041020 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a240c555-49f6-47cb-9a8e-7f43b03d2985-tmp\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.044389 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.044365 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfplp\" (UniqueName: \"kubernetes.io/projected/898634ce-48a8-473d-9809-2ad41c4fa11c-kube-api-access-qfplp\") pod \"aws-ebs-csi-driver-node-brw29\" (UID: \"898634ce-48a8-473d-9809-2ad41c4fa11c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.044656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.044636 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxp6w\" (UniqueName: \"kubernetes.io/projected/dcd9495b-b6b5-43a1-891c-d409dffab963-kube-api-access-rxp6w\") pod \"iptables-alerter-ss4ml\" (UID: \"dcd9495b-b6b5-43a1-891c-d409dffab963\") " pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.054742 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.054714 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn575\" (UniqueName: \"kubernetes.io/projected/601895bc-5e06-4dd8-bdf1-43babbb67529-kube-api-access-pn575\") pod \"node-resolver-wr9zd\" (UID: \"601895bc-5e06-4dd8-bdf1-43babbb67529\") " pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.054912 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.054889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbtfn\" (UniqueName: \"kubernetes.io/projected/e3e80296-decf-49a0-a5c9-71630ad65571-kube-api-access-bbtfn\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:15.055023 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.055002 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmfr8\" (UniqueName: \"kubernetes.io/projected/cdb94fad-2c12-4d5b-94ee-c31a4f061916-kube-api-access-vmfr8\") pod \"multus-25wmd\" (UID: \"cdb94fad-2c12-4d5b-94ee-c31a4f061916\") " pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.055814 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.055756 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kw7f\" (UniqueName: \"kubernetes.io/projected/a240c555-49f6-47cb-9a8e-7f43b03d2985-kube-api-access-9kw7f\") pod \"tuned-5nhb7\" (UID: \"a240c555-49f6-47cb-9a8e-7f43b03d2985\") " pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.055814 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.055769 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxvjr\" (UniqueName: \"kubernetes.io/projected/a6cd8d74-720e-4c12-b299-21b412df7af2-kube-api-access-vxvjr\") pod \"ovnkube-node-76p9p\" (UID: \"a6cd8d74-720e-4c12-b299-21b412df7af2\") " pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.077981 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.077924 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" event={"ID":"81b239ad80af5a4b339428b7893ca0f3","Type":"ContainerStarted","Data":"c0e27f9fa59cc5b597f97671d59893d9dbb329a8ce71ff61812d205e023e2d43"} Jul 07 10:27:15.078985 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.078960 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" event={"ID":"9585bff18ee532ee27e6d3ba6f76cb3f","Type":"ContainerStarted","Data":"6ac784f1f02308a44e69a0ee8ba78aded836908bb9e3e466b85c6e9fecfadfa7"} Jul 07 10:27:15.138967 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.138929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.138978 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-os-release\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55bcc27c-62e9-4172-8855-09ac0c8cb194-serviceca\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92bdf557-5f55-49e0-9c91-eb18897f1e92-konnectivity-ca\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139088 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92bdf557-5f55-49e0-9c91-eb18897f1e92-agent-certs\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.139149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-system-cni-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cnibin\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmnzp\" (UniqueName: \"kubernetes.io/projected/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-kube-api-access-xmnzp\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139289 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-799w7\" (UniqueName: \"kubernetes.io/projected/55bcc27c-62e9-4172-8855-09ac0c8cb194-kube-api-access-799w7\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:15.139390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bcc27c-62e9-4172-8855-09ac0c8cb194-host\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.139602 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.139432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bcc27c-62e9-4172-8855-09ac0c8cb194-host\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.140039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-binary-copy\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.140127 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140112 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-os-release\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.140175 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140162 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-system-cni-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.140239 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140226 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cnibin\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.140563 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140542 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.141000 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.140961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.141430 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.141215 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.141430 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.141302 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/92bdf557-5f55-49e0-9c91-eb18897f1e92-konnectivity-ca\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.141430 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.141373 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55bcc27c-62e9-4172-8855-09ac0c8cb194-serviceca\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.143368 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.143348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/92bdf557-5f55-49e0-9c91-eb18897f1e92-agent-certs\") pod \"konnectivity-agent-p7s6m\" (UID: \"92bdf557-5f55-49e0-9c91-eb18897f1e92\") " pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.146317 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.146299 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:15.146408 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.146321 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:15.146408 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.146331 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:15.146408 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.146386 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:15.646368537 +0000 UTC m=+3.106903263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:15.149923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.149865 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-799w7\" (UniqueName: \"kubernetes.io/projected/55bcc27c-62e9-4172-8855-09ac0c8cb194-kube-api-access-799w7\") pod \"node-ca-vfdgc\" (UID: \"55bcc27c-62e9-4172-8855-09ac0c8cb194\") " pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.150011 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.149988 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmnzp\" (UniqueName: \"kubernetes.io/projected/a6e669d0-aab7-4a43-ab0b-b134c2b6279c-kube-api-access-xmnzp\") pod \"multus-additional-cni-plugins-6xhsq\" (UID: \"a6e669d0-aab7-4a43-ab0b-b134c2b6279c\") " pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.230007 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.229967 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" Jul 07 10:27:15.235607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.235567 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 10:27:15.236653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.236635 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-25wmd" Jul 07 10:27:15.246411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.246392 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss4ml" Jul 07 10:27:15.253148 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.253130 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" Jul 07 10:27:15.261747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.261730 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:15.268353 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.268334 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr9zd" Jul 07 10:27:15.276180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.276164 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vfdgc" Jul 07 10:27:15.283900 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.283880 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:15.290073 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.290051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" Jul 07 10:27:15.542371 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.542293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:15.542371 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.542366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:15.542567 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.542442 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:15.542567 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.542527 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:16.542505362 +0000 UTC m=+4.003040104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:15.542567 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.542450 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:15.542713 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.542582 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:16.542566986 +0000 UTC m=+4.003101713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:15.743570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.743532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:15.743760 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.743731 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:15.743760 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.743761 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:15.743958 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.743776 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:15.743958 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:15.743850 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:16.743835699 +0000 UTC m=+4.204370422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:15.911070 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.911045 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb94fad_2c12_4d5b_94ee_c31a4f061916.slice/crio-7a7f626f5520a343fe3c60d02b52c226845f6b0d5a435bdbc00c815d8ec1f80c WatchSource:0}: Error finding container 7a7f626f5520a343fe3c60d02b52c226845f6b0d5a435bdbc00c815d8ec1f80c: Status 404 returned error can't find the container with id 7a7f626f5520a343fe3c60d02b52c226845f6b0d5a435bdbc00c815d8ec1f80c Jul 07 10:27:15.912820 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.912771 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92bdf557_5f55_49e0_9c91_eb18897f1e92.slice/crio-1162e3363fd3d5dafe0ab605ba62d6c2012dca2e038a25b49c17138229cc137b WatchSource:0}: Error finding container 1162e3363fd3d5dafe0ab605ba62d6c2012dca2e038a25b49c17138229cc137b: Status 404 returned error can't find the container with id 1162e3363fd3d5dafe0ab605ba62d6c2012dca2e038a25b49c17138229cc137b Jul 07 10:27:15.913993 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.913895 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda240c555_49f6_47cb_9a8e_7f43b03d2985.slice/crio-dad239b25b4ef3af3f105c71e226662a3e9416b9bd2a54b94e1eac6cd647a7ce WatchSource:0}: Error finding container dad239b25b4ef3af3f105c71e226662a3e9416b9bd2a54b94e1eac6cd647a7ce: Status 404 returned error can't find the container with id dad239b25b4ef3af3f105c71e226662a3e9416b9bd2a54b94e1eac6cd647a7ce Jul 07 10:27:15.916305 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.916263 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898634ce_48a8_473d_9809_2ad41c4fa11c.slice/crio-50a90f23c027752ec4e1e9d24aaa443a58295df9f28f30367094baafd15e6f69 WatchSource:0}: Error finding container 50a90f23c027752ec4e1e9d24aaa443a58295df9f28f30367094baafd15e6f69: Status 404 returned error can't find the container with id 50a90f23c027752ec4e1e9d24aaa443a58295df9f28f30367094baafd15e6f69 Jul 07 10:27:15.919179 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.919160 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e669d0_aab7_4a43_ab0b_b134c2b6279c.slice/crio-8cb60067f3d00a8066bc70aa7804ca12ce2971cc6edc09a1aafabbb3d9af50e1 WatchSource:0}: Error finding container 8cb60067f3d00a8066bc70aa7804ca12ce2971cc6edc09a1aafabbb3d9af50e1: Status 404 returned error can't find the container with id 8cb60067f3d00a8066bc70aa7804ca12ce2971cc6edc09a1aafabbb3d9af50e1 Jul 07 10:27:15.920157 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.920129 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601895bc_5e06_4dd8_bdf1_43babbb67529.slice/crio-00bee88dd7969118c92165f685d9b679f4a47469688d2c90a32f644839e24687 WatchSource:0}: Error finding container 00bee88dd7969118c92165f685d9b679f4a47469688d2c90a32f644839e24687: Status 404 returned error can't find the container with id 00bee88dd7969118c92165f685d9b679f4a47469688d2c90a32f644839e24687 Jul 07 10:27:15.921703 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.921469 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd9495b_b6b5_43a1_891c_d409dffab963.slice/crio-8d3317700d630bc6efb910a2ed3aa73ac9da12f8e2fc295df6d6da0d2033c0fa WatchSource:0}: Error finding container 8d3317700d630bc6efb910a2ed3aa73ac9da12f8e2fc295df6d6da0d2033c0fa: Status 404 returned error can't find the container with id 8d3317700d630bc6efb910a2ed3aa73ac9da12f8e2fc295df6d6da0d2033c0fa Jul 07 10:27:15.922378 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.922355 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6cd8d74_720e_4c12_b299_21b412df7af2.slice/crio-6df53e1ae8886188b1ba8b93db896b2466a645ffe8e6119bf077ec3209c6d652 WatchSource:0}: Error finding container 6df53e1ae8886188b1ba8b93db896b2466a645ffe8e6119bf077ec3209c6d652: Status 404 returned error can't find the container with id 6df53e1ae8886188b1ba8b93db896b2466a645ffe8e6119bf077ec3209c6d652 Jul 07 10:27:15.924164 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:15.924030 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bcc27c_62e9_4172_8855_09ac0c8cb194.slice/crio-fbe97dbf8ff2e3332a264f890ccb4f8de5e42834926446ac4e96f9200a16e695 WatchSource:0}: Error finding container fbe97dbf8ff2e3332a264f890ccb4f8de5e42834926446ac4e96f9200a16e695: Status 404 returned error can't find the container with id fbe97dbf8ff2e3332a264f890ccb4f8de5e42834926446ac4e96f9200a16e695 Jul 07 10:27:15.968307 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.968257 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 10:22:14 +0000 UTC" deadline="2027-12-24 23:52:56.837280991 +0000 UTC" Jul 07 10:27:15.968307 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:15.968306 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12853h25m40.868978802s" Jul 07 10:27:16.073141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.073116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:16.073141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.073133 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:16.073339 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.073115 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:16.073339 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.073233 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:16.073339 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.073297 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:16.073448 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.073381 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:16.081752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.081725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7s6m" event={"ID":"92bdf557-5f55-49e0-9c91-eb18897f1e92","Type":"ContainerStarted","Data":"1162e3363fd3d5dafe0ab605ba62d6c2012dca2e038a25b49c17138229cc137b"} Jul 07 10:27:16.084261 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.084232 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" event={"ID":"9585bff18ee532ee27e6d3ba6f76cb3f","Type":"ContainerStarted","Data":"4808ed13cc6e3e2b0d41513f00e8604b0e375eb6a0a4921f6a86994e9065b43e"} Jul 07 10:27:16.085462 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.085440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr9zd" event={"ID":"601895bc-5e06-4dd8-bdf1-43babbb67529","Type":"ContainerStarted","Data":"00bee88dd7969118c92165f685d9b679f4a47469688d2c90a32f644839e24687"} Jul 07 10:27:16.086733 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.086708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" event={"ID":"898634ce-48a8-473d-9809-2ad41c4fa11c","Type":"ContainerStarted","Data":"50a90f23c027752ec4e1e9d24aaa443a58295df9f28f30367094baafd15e6f69"} Jul 07 10:27:16.087911 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.087893 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" event={"ID":"a240c555-49f6-47cb-9a8e-7f43b03d2985","Type":"ContainerStarted","Data":"dad239b25b4ef3af3f105c71e226662a3e9416b9bd2a54b94e1eac6cd647a7ce"} Jul 07 10:27:16.089269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.089246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-25wmd" event={"ID":"cdb94fad-2c12-4d5b-94ee-c31a4f061916","Type":"ContainerStarted","Data":"7a7f626f5520a343fe3c60d02b52c226845f6b0d5a435bdbc00c815d8ec1f80c"} Jul 07 10:27:16.090906 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.090885 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vfdgc" event={"ID":"55bcc27c-62e9-4172-8855-09ac0c8cb194","Type":"ContainerStarted","Data":"fbe97dbf8ff2e3332a264f890ccb4f8de5e42834926446ac4e96f9200a16e695"} Jul 07 10:27:16.091948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.091929 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"6df53e1ae8886188b1ba8b93db896b2466a645ffe8e6119bf077ec3209c6d652"} Jul 07 10:27:16.093087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.093064 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss4ml" event={"ID":"dcd9495b-b6b5-43a1-891c-d409dffab963","Type":"ContainerStarted","Data":"8d3317700d630bc6efb910a2ed3aa73ac9da12f8e2fc295df6d6da0d2033c0fa"} Jul 07 10:27:16.094228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.094208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerStarted","Data":"8cb60067f3d00a8066bc70aa7804ca12ce2971cc6edc09a1aafabbb3d9af50e1"} Jul 07 10:27:16.104570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.104527 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-204.ec2.internal" podStartSLOduration=2.104514125 podStartE2EDuration="2.104514125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:16.104333444 +0000 UTC m=+3.564868213" watchObservedRunningTime="2026-07-07 10:27:16.104514125 +0000 UTC m=+3.565048869" Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.549461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.549593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.549719 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.549780 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:18.549762499 +0000 UTC m=+6.010297236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.550190 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:16.550278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.550238 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:18.55022344 +0000 UTC m=+6.010758169 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:16.752939 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.751643 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:16.752939 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.751887 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:16.752939 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.751915 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:16.752939 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.751928 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:16.752939 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:16.752009 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:18.751988971 +0000 UTC m=+6.212523697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:16.969234 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.969134 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 10:22:14 +0000 UTC" deadline="2028-04-10 09:46:50.554402814 +0000 UTC" Jul 07 10:27:16.969234 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:16.969197 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15431h19m33.58521011s" Jul 07 10:27:17.120441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:17.119865 2581 generic.go:356] "Generic (PLEG): container finished" podID="81b239ad80af5a4b339428b7893ca0f3" containerID="81e4afebd19a46a3902a7d50734dc779270adf52d8f9a03f80401ac72313aa71" exitCode=0 Jul 07 10:27:17.120441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:17.120002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" event={"ID":"81b239ad80af5a4b339428b7893ca0f3","Type":"ContainerDied","Data":"81e4afebd19a46a3902a7d50734dc779270adf52d8f9a03f80401ac72313aa71"} Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.073930 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.074053 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.074490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.074598 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.074659 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:18.074778 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.074729 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:18.145819 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.143641 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" event={"ID":"81b239ad80af5a4b339428b7893ca0f3","Type":"ContainerStarted","Data":"f346931d71a7636252e608ee7387d1ede23e6938fbbd2d4f6b95af9b13c0cd47"} Jul 07 10:27:18.160990 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.160921 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-204.ec2.internal" podStartSLOduration=4.160899561 podStartE2EDuration="4.160899561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:18.159954196 +0000 UTC m=+5.620488942" watchObservedRunningTime="2026-07-07 10:27:18.160899561 +0000 UTC m=+5.621434306" Jul 07 10:27:18.566661 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.566569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:18.566661 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.566649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:18.566944 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.566812 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:18.566944 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.566876 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:22.566857902 +0000 UTC m=+10.027392630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:18.567293 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.567273 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:18.567375 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.567330 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:22.567314562 +0000 UTC m=+10.027849312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:18.768840 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:18.768154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:18.768840 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.768353 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:18.768840 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.768376 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:18.768840 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.768392 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:18.768840 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:18.768452 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:22.768432318 +0000 UTC m=+10.228967058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:20.073005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:20.072974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:20.073433 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:20.073093 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:20.073433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:20.073140 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:20.073433 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:20.073183 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:20.073433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:20.073213 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:20.073433 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:20.073294 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.072946 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.073088 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.073500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.073596 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.073665 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:22.073847 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.073738 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:22.599753 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.599589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:22.599753 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.599675 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:22.600015 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.599763 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:22.600015 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.599835 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:22.600015 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.599858 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:30.599836312 +0000 UTC m=+18.060371071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:22.600015 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.599898 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:30.599879869 +0000 UTC m=+18.060414619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:22.800942 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:22.800903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:22.801138 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.801105 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:22.801138 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.801134 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:22.801254 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.801149 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:22.801254 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:22.801221 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:30.801200816 +0000 UTC m=+18.261735554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:24.073350 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:24.073316 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:24.073811 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:24.073438 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:24.073811 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:24.073322 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:24.073811 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:24.073507 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:24.073811 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:24.073620 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:24.073811 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:24.073692 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:26.073178 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:26.073141 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:26.073590 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:26.073196 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:26.073590 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:26.073158 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:26.073590 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:26.073285 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:26.073590 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:26.073399 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:26.073590 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:26.073489 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:28.073960 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:28.073919 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:28.074409 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:28.073987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:28.074409 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:28.073933 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:28.074409 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:28.074058 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:28.074409 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:28.074168 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:28.074409 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:28.074251 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:30.073604 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.073573 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:30.074172 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.073623 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:30.074172 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.073687 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:30.074172 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.073700 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:30.074172 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.073824 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:30.074172 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.073898 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:30.657609 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.657571 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:30.657781 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.657626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:30.657781 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.657724 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:30.657781 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.657724 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:30.657913 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.657789 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:46.657771212 +0000 UTC m=+34.118305949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:30.657913 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.657826 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:27:46.657816675 +0000 UTC m=+34.118351398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:30.858702 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:30.858616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:30.858879 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.858784 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:30.858879 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.858825 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:30.858879 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.858839 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:30.859011 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:30.858906 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:27:46.858891673 +0000 UTC m=+34.319426399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:32.073163 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:32.073129 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:32.073163 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:32.073161 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:32.073571 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:32.073241 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:32.073571 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:32.073266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:32.073571 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:32.073325 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:32.073571 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:32.073412 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:34.073234 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:34.073195 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:34.073697 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:34.073190 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:34.073697 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:34.073334 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:34.073697 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:34.073351 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:34.073697 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:34.073457 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:34.073697 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:34.073519 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:35.181868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.181411 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vfdgc" event={"ID":"55bcc27c-62e9-4172-8855-09ac0c8cb194","Type":"ContainerStarted","Data":"df229f69448024282df2e28452ca8086652929abf49557e6da367161bdbfbdf2"} Jul 07 10:27:35.183999 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.183877 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"99f24f393a658110305836d7543a2bbefbe3a2a8f5dafba2c98c269105157250"} Jul 07 10:27:35.183999 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.183944 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"72291cf1eede7d5af61a73b74b0fb2b99eb068c98a40d4f083b260a2ab38d3ab"} Jul 07 10:27:35.185726 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.185706 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerStarted","Data":"171c2dbb2cd63c0e918d931cc9357281237748ce99e1144123e8f01f76e8f691"} Jul 07 10:27:35.187499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.187480 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7s6m" event={"ID":"92bdf557-5f55-49e0-9c91-eb18897f1e92","Type":"ContainerStarted","Data":"bc7e990c2b925f0ed0686ce2460600f259e8bdaabe81c50a324828ae3f5fbb0c"} Jul 07 10:27:35.189439 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.189116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr9zd" event={"ID":"601895bc-5e06-4dd8-bdf1-43babbb67529","Type":"ContainerStarted","Data":"947fbe54ca8890724fd37ab14e74e2500f39ec4dae8fdc357c64669ca8b0c25e"} Jul 07 10:27:35.190835 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.190788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" event={"ID":"898634ce-48a8-473d-9809-2ad41c4fa11c","Type":"ContainerStarted","Data":"18b54998434a674e6109573af28ccb2b746055e77db33d014abb1be4b2a1f803"} Jul 07 10:27:35.192303 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.192283 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" event={"ID":"a240c555-49f6-47cb-9a8e-7f43b03d2985","Type":"ContainerStarted","Data":"9e305021be41e9bc5f7d6655a94a25ae72b0ff9e77073e11b7f7168793b091c9"} Jul 07 10:27:35.193972 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.193950 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-25wmd" event={"ID":"cdb94fad-2c12-4d5b-94ee-c31a4f061916","Type":"ContainerStarted","Data":"0126b53e666e79e0e99e421ebdc4632d52db018add2cb6a0a8bc08c7ea21d94b"} Jul 07 10:27:35.196447 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.196402 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vfdgc" podStartSLOduration=3.235600187 podStartE2EDuration="22.196386928s" totalImagesPullingTime="18.960786741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:35.196177621 +0000 UTC m=+22.656712366" watchObservedRunningTime="2026-07-07 10:27:35.196386928 +0000 UTC m=+22.656921674" Jul 07 10:27:35.212628 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.212570 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5nhb7" podStartSLOduration=3.241233414 podStartE2EDuration="22.212549488s" totalImagesPullingTime="18.971316074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:35.212288489 +0000 UTC m=+22.672823254" watchObservedRunningTime="2026-07-07 10:27:35.212549488 +0000 UTC m=+22.673084234" Jul 07 10:27:35.227520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.227475 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wr9zd" podStartSLOduration=3.265048417 podStartE2EDuration="22.227459952s" totalImagesPullingTime="18.962411535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:35.226518819 +0000 UTC m=+22.687053563" watchObservedRunningTime="2026-07-07 10:27:35.227459952 +0000 UTC m=+22.687994696" Jul 07 10:27:35.247587 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.247535 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p7s6m" podStartSLOduration=7.675935111 podStartE2EDuration="22.24751571s" totalImagesPullingTime="14.571580599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:35.247462159 +0000 UTC m=+22.707996917" watchObservedRunningTime="2026-07-07 10:27:35.24751571 +0000 UTC m=+22.708050482" Jul 07 10:27:35.295735 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:35.295688 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-25wmd" podStartSLOduration=3.262056359 podStartE2EDuration="22.295671239s" totalImagesPullingTime="19.03361488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:35.295654608 +0000 UTC m=+22.756189352" watchObservedRunningTime="2026-07-07 10:27:35.295671239 +0000 UTC m=+22.756206273" Jul 07 10:27:36.073019 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.072982 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:36.073019 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.073020 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:36.073272 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.072991 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:36.073272 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:36.073104 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:36.073272 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:36.073155 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:36.073434 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:36.073253 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:36.201165 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.201075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"d9b5d9e3e616666f7d115acb91dc8e83650dbf03eb3c723f4963715152f8e597"} Jul 07 10:27:36.201165 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.201162 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"1bd1030f395160ec95dcbf90c3fb74cdb90fd08dfacda5acef042bd6eecf941b"} Jul 07 10:27:36.201641 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.201212 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"d77d5c7ed8cbaaa4e1a73304328be2036e096d5261ee36316728506b08fca8b6"} Jul 07 10:27:36.201641 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.201257 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"9c9f2621bbcd1d203af32bd33ea2406791f0d1048ee5c68faceff0fc6e267c12"} Jul 07 10:27:36.202673 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.202641 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="171c2dbb2cd63c0e918d931cc9357281237748ce99e1144123e8f01f76e8f691" exitCode=0 Jul 07 10:27:36.202811 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.202684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"171c2dbb2cd63c0e918d931cc9357281237748ce99e1144123e8f01f76e8f691"} Jul 07 10:27:36.774962 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.774915 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 10:27:36.995926 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.995760 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T10:27:36.77493805Z","UUID":"6136376a-ae47-488b-b388-27517751a7c9","Handler":null,"Name":"","Endpoint":""} Jul 07 10:27:36.997571 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.997552 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 10:27:36.997685 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:36.997591 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 10:27:37.206247 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:37.206210 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss4ml" event={"ID":"dcd9495b-b6b5-43a1-891c-d409dffab963","Type":"ContainerStarted","Data":"1b8d6b2bb76b4853158a7a13c53e88821d689862d55248e6449e1471af841d8f"} Jul 07 10:27:37.208112 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:37.208084 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" event={"ID":"898634ce-48a8-473d-9809-2ad41c4fa11c","Type":"ContainerStarted","Data":"ae14d09213d06783eed76a8467b3faa8b2a756eda54d9b09fc0e917343abb376"} Jul 07 10:27:37.222873 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:37.222826 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ss4ml" podStartSLOduration=5.259213747 podStartE2EDuration="24.222781344s" totalImagesPullingTime="18.963567597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:37.222542189 +0000 UTC m=+24.683076935" watchObservedRunningTime="2026-07-07 10:27:37.222781344 +0000 UTC m=+24.683316088" Jul 07 10:27:38.073243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.073212 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:38.073386 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.073247 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:38.073386 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.073314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:38.073386 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:38.073327 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:38.073481 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:38.073409 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:38.073522 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:38.073503 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:38.212449 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.212411 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" event={"ID":"898634ce-48a8-473d-9809-2ad41c4fa11c","Type":"ContainerStarted","Data":"d45d22f3bc549d56633e84b350d76e01b368d310cde619abcf498e8f0385dcab"} Jul 07 10:27:38.215397 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.215361 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"f7f55fdf37ffee2fddd24797c289371f766093fda333d4d1eaee7057e52f0309"} Jul 07 10:27:38.235784 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.235736 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-brw29" podStartSLOduration=8.009418429 podStartE2EDuration="25.235721334s" totalImagesPullingTime="17.226302905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:38.235238153 +0000 UTC m=+25.695772899" watchObservedRunningTime="2026-07-07 10:27:38.235721334 +0000 UTC m=+25.696256079" Jul 07 10:27:38.396544 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.396454 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:38.397311 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:38.397289 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:39.218691 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:39.218485 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:39.219269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:39.219245 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p7s6m" Jul 07 10:27:40.074040 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:40.073964 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:40.074040 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:40.074000 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:40.074272 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:40.073959 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:40.074272 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:40.074091 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:40.074272 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:40.074192 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:40.074409 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:40.074268 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:41.226279 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.225997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" event={"ID":"a6cd8d74-720e-4c12-b299-21b412df7af2","Type":"ContainerStarted","Data":"b6ece4ab47cfe461a09fb3af0d99612c500286324467e9b4aee344d5325e9914"} Jul 07 10:27:41.226279 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.226279 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:41.227154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.226322 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:41.227154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.226353 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:41.229517 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.229482 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="e0cd58a1eac1989c898939a155aef4be034bbaca32b5e03b68b25b184754be99" exitCode=0 Jul 07 10:27:41.229636 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.229533 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"e0cd58a1eac1989c898939a155aef4be034bbaca32b5e03b68b25b184754be99"} Jul 07 10:27:41.252438 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.252410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:41.252568 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.252515 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:27:41.258003 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:41.257964 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" podStartSLOduration=9.226448917999999 podStartE2EDuration="28.257951547s" totalImagesPullingTime="19.031502629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:41.257577507 +0000 UTC m=+28.718112251" watchObservedRunningTime="2026-07-07 10:27:41.257951547 +0000 UTC m=+28.718486292" Jul 07 10:27:42.073644 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.073576 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:42.073901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.073577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:42.073901 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.073715 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:42.073901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.073586 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:42.073901 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.073792 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:42.073901 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.073876 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:42.730086 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.730055 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9wvp"] Jul 07 10:27:42.730429 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.730154 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:42.730429 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.730245 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:42.736684 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.736663 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdbtk"] Jul 07 10:27:42.736824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.736735 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:42.736890 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.736855 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:42.739280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.739259 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hgc7q"] Jul 07 10:27:42.739381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:42.739348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:42.739467 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:42.739447 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:43.239567 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:43.239530 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="9e776f328d2062ae79c2cf023097df90c8ae510721feae66468fa51affba83dc" exitCode=0 Jul 07 10:27:43.239756 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:43.239625 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"9e776f328d2062ae79c2cf023097df90c8ae510721feae66468fa51affba83dc"} Jul 07 10:27:44.073443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:44.073411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:44.073944 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:44.073411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:44.073944 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:44.073549 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:44.073944 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:44.073627 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:45.073735 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:45.073502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:45.074115 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:45.073763 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:45.247221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:45.247189 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="236cf2a42751346248d3cafecb2d6a414f33f6ce3b7db84417e0ce705ec733b3" exitCode=0 Jul 07 10:27:45.247380 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:45.247252 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"236cf2a42751346248d3cafecb2d6a414f33f6ce3b7db84417e0ce705ec733b3"} Jul 07 10:27:46.073080 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:46.073039 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:46.073284 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:46.073039 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:46.073284 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.073186 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:46.073284 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.073253 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:46.679172 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:46.679142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:46.679698 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:46.679197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:46.679698 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.679309 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:46.679698 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.679314 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:46.679698 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.679372 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret podName:a2087d4e-9d80-4d94-b537-077d2901c6fa nodeName:}" failed. No retries permitted until 2026-07-07 10:28:18.679356649 +0000 UTC m=+66.139891386 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret") pod "global-pull-secret-syncer-h9wvp" (UID: "a2087d4e-9d80-4d94-b537-077d2901c6fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 10:27:46.679698 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.679388 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:18.679379873 +0000 UTC m=+66.139914599 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 10:27:46.880551 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:46.880440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:46.880691 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.880610 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 10:27:46.880691 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.880633 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 10:27:46.880691 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.880643 2581 projected.go:196] Error preparing data for projected volume kube-api-access-22qh6 for pod openshift-network-diagnostics/network-check-target-sdbtk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:46.880782 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:46.880693 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6 podName:27ff8724-8b54-4013-99a7-f3c09dac2063 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:18.880680353 +0000 UTC m=+66.341215079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-22qh6" (UniqueName: "kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6") pod "network-check-target-sdbtk" (UID: "27ff8724-8b54-4013-99a7-f3c09dac2063") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 10:27:47.073390 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:47.073354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:47.073570 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:47.073496 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hgc7q" podUID="e3e80296-decf-49a0-a5c9-71630ad65571" Jul 07 10:27:48.073871 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.073841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:48.074315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.073840 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:48.074315 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:48.073958 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdbtk" podUID="27ff8724-8b54-4013-99a7-f3c09dac2063" Jul 07 10:27:48.074315 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:48.074055 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wvp" podUID="a2087d4e-9d80-4d94-b537-077d2901c6fa" Jul 07 10:27:48.872369 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.872336 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-204.ec2.internal" event="NodeReady" Jul 07 10:27:48.872520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.872476 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 10:27:48.919518 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.919479 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7ltr2"] Jul 07 10:27:48.922441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.922418 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:48.926435 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.926416 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 10:27:48.927104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.926954 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 10:27:48.927219 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.927197 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lkwtd"] Jul 07 10:27:48.927267 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.927219 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5jm\"" Jul 07 10:27:48.929296 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.929009 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:48.931927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.931715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 10:27:48.931927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.931773 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 10:27:48.931927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.931862 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 10:27:48.932342 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.932323 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mdcws\"" Jul 07 10:27:48.937452 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.937430 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7ltr2"] Jul 07 10:27:48.946783 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:48.946751 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lkwtd"] Jul 07 10:27:49.073783 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.073746 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:27:49.077082 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.077055 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 10:27:49.077568 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.077087 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-68x57\"" Jul 07 10:27:49.096441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-tmp-dir\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.096592 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096460 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-cert\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.096592 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-config-volume\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.096592 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096580 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.096703 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnwql\" (UniqueName: \"kubernetes.io/projected/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-kube-api-access-lnwql\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.096703 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.096648 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nj6c\" (UniqueName: \"kubernetes.io/projected/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-kube-api-access-8nj6c\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.197489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197404 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-tmp-dir\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.197489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197456 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-cert\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.197692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-config-volume\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.197692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.197692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnwql\" (UniqueName: \"kubernetes.io/projected/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-kube-api-access-lnwql\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.197692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nj6c\" (UniqueName: \"kubernetes.io/projected/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-kube-api-access-8nj6c\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.197865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.197833 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-tmp-dir\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.197963 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:49.197935 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:27:49.198041 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:49.198019 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:27:49.697995701 +0000 UTC m=+37.158530432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:27:49.198175 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.198134 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-config-volume\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.202506 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.202484 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-cert\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.208335 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.208309 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnwql\" (UniqueName: \"kubernetes.io/projected/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-kube-api-access-lnwql\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.208696 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.208673 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nj6c\" (UniqueName: \"kubernetes.io/projected/42d87c1b-6ae0-4156-87ff-bdb1b0c77b76-kube-api-access-8nj6c\") pod \"ingress-canary-lkwtd\" (UID: \"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76\") " pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.241746 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.241716 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lkwtd" Jul 07 10:27:49.702568 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:49.702534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:49.702747 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:49.702708 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:27:49.702821 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:49.702787 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:27:50.70276757 +0000 UTC m=+38.163302306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:27:50.073910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.073772 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:27:50.073910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.073790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:27:50.078055 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.077852 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdqsn\"" Jul 07 10:27:50.078055 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.077860 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 10:27:50.078055 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.077949 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 10:27:50.078055 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.077953 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 10:27:50.216852 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.216584 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lkwtd"] Jul 07 10:27:50.220907 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:27:50.220875 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d87c1b_6ae0_4156_87ff_bdb1b0c77b76.slice/crio-a4bb38e934385de083cceec44ca2e9f5b276d58b7405d81403045f08ccd5969d WatchSource:0}: Error finding container a4bb38e934385de083cceec44ca2e9f5b276d58b7405d81403045f08ccd5969d: Status 404 returned error can't find the container with id a4bb38e934385de083cceec44ca2e9f5b276d58b7405d81403045f08ccd5969d Jul 07 10:27:50.261974 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.261938 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerStarted","Data":"ed88c62c46cb8e845a5533b98213d14951f92ebf085788cf2365db5fc7557c91"} Jul 07 10:27:50.263067 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.263039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lkwtd" event={"ID":"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76","Type":"ContainerStarted","Data":"a4bb38e934385de083cceec44ca2e9f5b276d58b7405d81403045f08ccd5969d"} Jul 07 10:27:50.711472 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:50.711436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:50.711643 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:50.711597 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:27:50.711682 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:50.711674 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:27:52.711652221 +0000 UTC m=+40.172186952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:27:51.268516 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:51.268481 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="ed88c62c46cb8e845a5533b98213d14951f92ebf085788cf2365db5fc7557c91" exitCode=0 Jul 07 10:27:51.268938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:51.268536 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"ed88c62c46cb8e845a5533b98213d14951f92ebf085788cf2365db5fc7557c91"} Jul 07 10:27:52.275173 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:52.275135 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6e669d0-aab7-4a43-ab0b-b134c2b6279c" containerID="494928583c7a3024bf1ed8af1b53f205fe86f354afc27d340251a8ab3b71a414" exitCode=0 Jul 07 10:27:52.275632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:52.275183 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerDied","Data":"494928583c7a3024bf1ed8af1b53f205fe86f354afc27d340251a8ab3b71a414"} Jul 07 10:27:52.430457 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:52.430372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:52.539374 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:52.539349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:52.727696 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:52.727668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:52.727867 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:52.727848 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:27:52.727933 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:52.727923 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:27:56.72790772 +0000 UTC m=+44.188442447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:27:53.190653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.190621 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:53.253647 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.253615 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:53.280498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.280466 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" event={"ID":"a6e669d0-aab7-4a43-ab0b-b134c2b6279c","Type":"ContainerStarted","Data":"ee242a4e3105262048b5a24b556fcdbf92de73cc5c914310156e95aec2306247"} Jul 07 10:27:53.281848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.281826 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lkwtd" event={"ID":"42d87c1b-6ae0-4156-87ff-bdb1b0c77b76","Type":"ContainerStarted","Data":"33905ca7d4ade30fd7b6b5dc84610574bfa61b7dea80b06322272e23d35b941e"} Jul 07 10:27:53.305257 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.305206 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6xhsq" podStartSLOduration=9.062800942 podStartE2EDuration="40.305186484s" totalImagesPullingTime="31.242385542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:53.303521318 +0000 UTC m=+40.764056086" watchObservedRunningTime="2026-07-07 10:27:53.305186484 +0000 UTC m=+40.765721229" Jul 07 10:27:53.319838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.319765 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lkwtd" podStartSLOduration=2.872481789 podStartE2EDuration="5.319750232s" totalImagesPullingTime="2.447268443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:27:53.319240426 +0000 UTC m=+40.779775173" watchObservedRunningTime="2026-07-07 10:27:53.319750232 +0000 UTC m=+40.780284977" Jul 07 10:27:53.450496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.450417 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:53.512110 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.512074 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:53.839015 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.838924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:53.899278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:53.899248 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:27:56.756545 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:27:56.756505 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:27:56.756982 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:56.756674 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:27:56.756982 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:27:56.756751 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:28:04.75672484 +0000 UTC m=+52.217259567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:28:00.966134 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:00.966094 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:04.262292 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.262253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:04.327687 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.327617 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:04.815757 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.815723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:04.815936 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:04.815863 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 10:28:04.815936 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:04.815917 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls podName:b127bf4a-b560-4b09-8f9a-21fe84bc3dfe nodeName:}" failed. No retries permitted until 2026-07-07 10:28:20.815902873 +0000 UTC m=+68.276437597 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls") pod "dns-default-7ltr2" (UID: "b127bf4a-b560-4b09-8f9a-21fe84bc3dfe") : secret "dns-default-metrics-tls" not found Jul 07 10:28:04.877762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.877733 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x"] Jul 07 10:28:04.890030 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.890008 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" Jul 07 10:28:04.890344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.890317 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x"] Jul 07 10:28:04.893161 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.893141 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:28:04.893282 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.893187 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 10:28:04.894221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.894199 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xjqfr\"" Jul 07 10:28:04.986445 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:04.986409 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl"] Jul 07 10:28:05.007790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.007759 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb"] Jul 07 10:28:05.007947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.007885 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.010269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.010244 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 10:28:05.010436 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.010291 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bhcmw\"" Jul 07 10:28:05.010541 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.010526 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 10:28:05.010627 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.010533 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 10:28:05.010792 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.010777 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:28:05.017039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.017008 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb6b8\" (UniqueName: \"kubernetes.io/projected/294c4fe3-fc00-4825-b275-ad88fa4199c2-kube-api-access-mb6b8\") pod \"volume-data-source-validator-5d456fdbcd-vgx6x\" (UID: \"294c4fe3-fc00-4825-b275-ad88fa4199c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" Jul 07 10:28:05.025860 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.025837 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tf2f8"] Jul 07 10:28:05.025964 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.025945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.028570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.028552 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 10:28:05.028996 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.028982 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 10:28:05.029128 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.029107 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:28:05.029210 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.029163 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 10:28:05.029210 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.029169 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-j9xj5\"" Jul 07 10:28:05.052017 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.051991 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68d68d4cdf-ghwnt"] Jul 07 10:28:05.052142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.052101 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.054763 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.054743 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 10:28:05.054939 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.054839 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 10:28:05.054992 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.054974 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tbqnk\"" Jul 07 10:28:05.055164 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.055147 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 10:28:05.055298 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.055280 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 10:28:05.059628 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.059604 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 10:28:05.076111 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.076050 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl"] Jul 07 10:28:05.076219 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.076171 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.078829 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.078791 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 10:28:05.079046 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079024 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 10:28:05.079114 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079049 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tzdlj\"" Jul 07 10:28:05.079114 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079072 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 10:28:05.079211 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079051 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 10:28:05.079334 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079319 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 10:28:05.079393 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.079338 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 10:28:05.080033 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.080014 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb"] Jul 07 10:28:05.080135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.080054 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tf2f8"] Jul 07 10:28:05.080135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.080069 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68d68d4cdf-ghwnt"] Jul 07 10:28:05.117498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.117472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01eb5e09-8da9-4011-a831-235ee4ba0ce2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.117498 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.117501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01eb5e09-8da9-4011-a831-235ee4ba0ce2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.117729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.117524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxrxs\" (UniqueName: \"kubernetes.io/projected/01eb5e09-8da9-4011-a831-235ee4ba0ce2-kube-api-access-wxrxs\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.117729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.117653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6b8\" (UniqueName: \"kubernetes.io/projected/294c4fe3-fc00-4825-b275-ad88fa4199c2-kube-api-access-mb6b8\") pod \"volume-data-source-validator-5d456fdbcd-vgx6x\" (UID: \"294c4fe3-fc00-4825-b275-ad88fa4199c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" Jul 07 10:28:05.135614 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.135583 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6b8\" (UniqueName: \"kubernetes.io/projected/294c4fe3-fc00-4825-b275-ad88fa4199c2-kube-api-access-mb6b8\") pod \"volume-data-source-validator-5d456fdbcd-vgx6x\" (UID: \"294c4fe3-fc00-4825-b275-ad88fa4199c2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" Jul 07 10:28:05.169621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.169588 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:05.199674 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.199644 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" Jul 07 10:28:05.218981 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.218948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01eb5e09-8da9-4011-a831-235ee4ba0ce2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.218981 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.218984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01eb5e09-8da9-4011-a831-235ee4ba0ce2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.219170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219004 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxrxs\" (UniqueName: \"kubernetes.io/projected/01eb5e09-8da9-4011-a831-235ee4ba0ce2-kube-api-access-wxrxs\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.219170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219032 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-default-certificate\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.219170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219080 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-tmp\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219308 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219237 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/588b9ad5-e0c5-4fbf-9771-32305e6445c6-config\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.219308 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219294 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjnsb\" (UniqueName: \"kubernetes.io/projected/ec72510f-f217-42bf-90a6-05669e0f08fb-kube-api-access-mjnsb\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219403 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588b9ad5-e0c5-4fbf-9771-32305e6445c6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.219438 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219405 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rgdh\" (UniqueName: \"kubernetes.io/projected/588b9ad5-e0c5-4fbf-9771-32305e6445c6-kube-api-access-4rgdh\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.219470 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219506 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219479 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec72510f-f217-42bf-90a6-05669e0f08fb-serving-cert\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219555 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-stats-auth\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.219555 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.219656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219554 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01eb5e09-8da9-4011-a831-235ee4ba0ce2-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.219656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219572 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-snapshots\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219612 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.219656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219637 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.219821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.219662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65vg7\" (UniqueName: \"kubernetes.io/projected/438503da-c32e-49be-9296-4c1beb56f484-kube-api-access-65vg7\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.221422 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.221401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01eb5e09-8da9-4011-a831-235ee4ba0ce2-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.228069 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.228042 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxrxs\" (UniqueName: \"kubernetes.io/projected/01eb5e09-8da9-4011-a831-235ee4ba0ce2-kube-api-access-wxrxs\") pod \"kube-storage-version-migrator-operator-846bf668f7-v22dl\" (UID: \"01eb5e09-8da9-4011-a831-235ee4ba0ce2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.249276 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.249246 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:05.315173 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.315139 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:05.317874 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.317850 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" Jul 07 10:28:05.320099 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320074 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjnsb\" (UniqueName: \"kubernetes.io/projected/ec72510f-f217-42bf-90a6-05669e0f08fb-kube-api-access-mjnsb\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588b9ad5-e0c5-4fbf-9771-32305e6445c6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.320214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rgdh\" (UniqueName: \"kubernetes.io/projected/588b9ad5-e0c5-4fbf-9771-32305e6445c6-kube-api-access-4rgdh\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.320214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec72510f-f217-42bf-90a6-05669e0f08fb-serving-cert\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-stats-auth\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-snapshots\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65vg7\" (UniqueName: \"kubernetes.io/projected/438503da-c32e-49be-9296-4c1beb56f484-kube-api-access-65vg7\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-default-certificate\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.320411 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.320407 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:05.320777 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320424 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-tmp\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.320777 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.320461 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:05.820446192 +0000 UTC m=+53.280980915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:05.320777 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/588b9ad5-e0c5-4fbf-9771-32305e6445c6-config\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.320777 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.320538 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:05.820522408 +0000 UTC m=+53.281057153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:05.320777 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.320760 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-tmp\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.321474 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.321126 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec72510f-f217-42bf-90a6-05669e0f08fb-snapshots\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.321474 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.321164 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/588b9ad5-e0c5-4fbf-9771-32305e6445c6-config\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.321474 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.321252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.321682 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.321618 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec72510f-f217-42bf-90a6-05669e0f08fb-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.322777 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.322757 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588b9ad5-e0c5-4fbf-9771-32305e6445c6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.323515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.323493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec72510f-f217-42bf-90a6-05669e0f08fb-serving-cert\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.323818 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.323775 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-default-certificate\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.324004 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.323984 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-stats-auth\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.331213 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.331152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjnsb\" (UniqueName: \"kubernetes.io/projected/ec72510f-f217-42bf-90a6-05669e0f08fb-kube-api-access-mjnsb\") pod \"insights-operator-5c847dcfc4-tf2f8\" (UID: \"ec72510f-f217-42bf-90a6-05669e0f08fb\") " pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.331937 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.331907 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65vg7\" (UniqueName: \"kubernetes.io/projected/438503da-c32e-49be-9296-4c1beb56f484-kube-api-access-65vg7\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.332064 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.332045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rgdh\" (UniqueName: \"kubernetes.io/projected/588b9ad5-e0c5-4fbf-9771-32305e6445c6-kube-api-access-4rgdh\") pod \"service-ca-operator-5d46d9ffd4-jbgzb\" (UID: \"588b9ad5-e0c5-4fbf-9771-32305e6445c6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.333866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.333844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" Jul 07 10:28:05.344197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.344172 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x"] Jul 07 10:28:05.360905 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.360878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" Jul 07 10:28:05.479783 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.479616 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl"] Jul 07 10:28:05.483074 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:05.483042 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01eb5e09_8da9_4011_a831_235ee4ba0ce2.slice/crio-6b4b8af3d32e197317d4dd5623e42186c047dd56b9c6fd839e300f46307f48b2 WatchSource:0}: Error finding container 6b4b8af3d32e197317d4dd5623e42186c047dd56b9c6fd839e300f46307f48b2: Status 404 returned error can't find the container with id 6b4b8af3d32e197317d4dd5623e42186c047dd56b9c6fd839e300f46307f48b2 Jul 07 10:28:05.492670 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.492637 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb"] Jul 07 10:28:05.497123 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:05.497092 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588b9ad5_e0c5_4fbf_9771_32305e6445c6.slice/crio-76944760ec08b7ffbb85d8c4e118f3924db054e33232a2cbf7ae1620527c92d0 WatchSource:0}: Error finding container 76944760ec08b7ffbb85d8c4e118f3924db054e33232a2cbf7ae1620527c92d0: Status 404 returned error can't find the container with id 76944760ec08b7ffbb85d8c4e118f3924db054e33232a2cbf7ae1620527c92d0 Jul 07 10:28:05.530359 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.530330 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-tf2f8"] Jul 07 10:28:05.825109 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.825068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.825278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:05.825120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:05.825278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.825218 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:05.825278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.825250 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:06.825233658 +0000 UTC m=+54.285768383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:05.825278 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:05.825274 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:06.825262955 +0000 UTC m=+54.285797681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:06.319968 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.319924 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" event={"ID":"588b9ad5-e0c5-4fbf-9771-32305e6445c6","Type":"ContainerStarted","Data":"76944760ec08b7ffbb85d8c4e118f3924db054e33232a2cbf7ae1620527c92d0"} Jul 07 10:28:06.321906 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.321830 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" event={"ID":"294c4fe3-fc00-4825-b275-ad88fa4199c2","Type":"ContainerStarted","Data":"2f92540fa22a37049868cf2f4d38579be1734d7c72f010cf5a668d15f31ad86d"} Jul 07 10:28:06.323743 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.323688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" event={"ID":"ec72510f-f217-42bf-90a6-05669e0f08fb","Type":"ContainerStarted","Data":"fb6597ac7059beb0bb362847faf26a13cd17d6d47abaf5ccbc81c2fa93c71deb"} Jul 07 10:28:06.325085 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.325048 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" event={"ID":"01eb5e09-8da9-4011-a831-235ee4ba0ce2","Type":"ContainerStarted","Data":"6b4b8af3d32e197317d4dd5623e42186c047dd56b9c6fd839e300f46307f48b2"} Jul 07 10:28:06.835623 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.835583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:06.835960 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:06.835644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:06.835960 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:06.835829 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:08.835790663 +0000 UTC m=+56.296325392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:06.836087 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:06.836050 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:06.836141 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:06.836121 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:08.836104338 +0000 UTC m=+56.296639066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:07.328543 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:07.328516 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" event={"ID":"294c4fe3-fc00-4825-b275-ad88fa4199c2","Type":"ContainerStarted","Data":"1e959eb59f7c48eff4dbcb3f361005187686c5aa51a3f823757cc33504517e49"} Jul 07 10:28:08.220721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.220679 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:08.281233 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.281194 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:08.347922 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.347857 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-vgx6x" podStartSLOduration=2.495565657 podStartE2EDuration="4.347837313s" totalImagesPullingTime="1.852271656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:08.347369205 +0000 UTC m=+55.807903977" watchObservedRunningTime="2026-07-07 10:28:08.347837313 +0000 UTC m=+55.808372057" Jul 07 10:28:08.430624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.430586 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:08.490448 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.490351 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:08.853549 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.853454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:08.853549 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:08.853508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:08.853827 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:08.853632 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:08.853827 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:08.853648 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:12.853632933 +0000 UTC m=+60.314167669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:08.853827 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:08.853695 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:12.853677602 +0000 UTC m=+60.314212329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:09.338282 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:09.338241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" event={"ID":"ec72510f-f217-42bf-90a6-05669e0f08fb","Type":"ContainerStarted","Data":"e51148ce28693364d2cda9c5f8cddb9b84a15f15f7ab62b8a9689acab5c8243b"} Jul 07 10:28:09.339872 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:09.339837 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" event={"ID":"01eb5e09-8da9-4011-a831-235ee4ba0ce2","Type":"ContainerStarted","Data":"2d414e496568ad9218a080e0890ddedaf6c764106deab7793981b4f2d6518165"} Jul 07 10:28:09.341405 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:09.341380 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" event={"ID":"588b9ad5-e0c5-4fbf-9771-32305e6445c6","Type":"ContainerStarted","Data":"8d502af8c01dc82a43c7e59cd602a1c550e5749e3f839c439c58f8b2d8ce4659"} Jul 07 10:28:09.356306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:09.356247 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" podStartSLOduration=1.96284959 podStartE2EDuration="5.356226703s" totalImagesPullingTime="3.393377113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:09.355560269 +0000 UTC m=+56.816095047" watchObservedRunningTime="2026-07-07 10:28:09.356226703 +0000 UTC m=+56.816761430" Jul 07 10:28:09.371306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:09.371249 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" podStartSLOduration=1.920905866 podStartE2EDuration="5.371226255s" totalImagesPullingTime="3.450320389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:09.370345795 +0000 UTC m=+56.830880554" watchObservedRunningTime="2026-07-07 10:28:09.371226255 +0000 UTC m=+56.831761001" Jul 07 10:28:10.362909 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.362840 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" podStartSLOduration=2.924537007 podStartE2EDuration="6.362792349s" totalImagesPullingTime="3.438255342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:09.392147602 +0000 UTC m=+56.852682326" watchObservedRunningTime="2026-07-07 10:28:10.362792349 +0000 UTC m=+57.823327095" Jul 07 10:28:10.363687 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.363661 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8qswp"] Jul 07 10:28:10.367130 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.367110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.382481 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.382456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt2hz\"" Jul 07 10:28:10.401924 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.401899 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 10:28:10.423000 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.422970 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 10:28:10.458515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.458479 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8qswp"] Jul 07 10:28:10.465503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.465477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d34c59e-c9d0-41af-88ea-fd845c1689de-data-volume\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.465675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.465579 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rjf\" (UniqueName: \"kubernetes.io/projected/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-api-access-v4rjf\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.465675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.465617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.465675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.465643 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.465785 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.465756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d34c59e-c9d0-41af-88ea-fd845c1689de-crio-socket\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.520052 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.519992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:10.566221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566182 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d34c59e-c9d0-41af-88ea-fd845c1689de-data-volume\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rjf\" (UniqueName: \"kubernetes.io/projected/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-api-access-v4rjf\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566312 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d34c59e-c9d0-41af-88ea-fd845c1689de-crio-socket\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:10.566415 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:10.566437 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566460 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5d34c59e-c9d0-41af-88ea-fd845c1689de-crio-socket\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:10.566484 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:11.0664668 +0000 UTC m=+58.527001525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : configmap "kube-rbac-proxy" not found Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:10.566499 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:11.066492546 +0000 UTC m=+58.527027273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:10.566596 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.566532 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d34c59e-c9d0-41af-88ea-fd845c1689de-data-volume\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.577769 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.577738 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rjf\" (UniqueName: \"kubernetes.io/projected/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-api-access-v4rjf\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:10.671362 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.671324 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:10.772180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:10.772150 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.044167 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.044072 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.071198 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.071162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:11.071198 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.071200 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:11.071424 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:11.071270 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 10:28:11.071424 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:11.071325 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:12.071309557 +0000 UTC m=+59.531844280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : configmap "kube-rbac-proxy" not found Jul 07 10:28:11.071424 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:11.071332 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:11.071424 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:11.071393 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:12.071374119 +0000 UTC m=+59.531908842 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:11.100526 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.100486 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.311678 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.311585 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.415748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.415722 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.601916 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.601829 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:11.654980 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:11.654943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:12.081095 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.081058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:12.081095 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.081095 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:12.081310 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.081187 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 10:28:12.081310 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.081210 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:12.081310 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.081253 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:14.081237696 +0000 UTC m=+61.541772423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : configmap "kube-rbac-proxy" not found Jul 07 10:28:12.081310 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.081266 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:14.081260248 +0000 UTC m=+61.541794971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:12.559778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.559747 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr9zd_601895bc-5e06-4dd8-bdf1-43babbb67529/dns-node-resolver/0.log" Jul 07 10:28:12.886894 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.886861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:12.887043 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.886910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:12.887043 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.887012 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:12.887112 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.887041 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:20.887019885 +0000 UTC m=+68.347554610 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:12.887151 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:12.887121 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:20.887112518 +0000 UTC m=+68.347647240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:12.963297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:12.963259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:13.026412 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.026371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:13.251204 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.251134 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-76p9p" Jul 07 10:28:13.410147 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.410112 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:13.516675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.516577 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:13.560945 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.560921 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vfdgc_55bcc27c-62e9-4172-8855-09ac0c8cb194/node-ca/0.log" Jul 07 10:28:13.946369 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:13.946335 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:14.098971 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:14.098929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:14.098971 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:14.098968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:14.099176 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:14.099036 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 10:28:14.099176 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:14.099081 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:14.099176 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:14.099090 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:18.099076171 +0000 UTC m=+65.559610898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : configmap "kube-rbac-proxy" not found Jul 07 10:28:14.099176 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:14.099153 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:18.099133959 +0000 UTC m=+65.559668683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:14.164244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:14.164216 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lkwtd_42d87c1b-6ae0-4156-87ff-bdb1b0c77b76/serve-healthcheck-canary/0.log" Jul 07 10:28:16.615992 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:16.615956 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:16.747652 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:16.747620 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:16.848402 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:16.848370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:16.949727 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:16.949687 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:17.053919 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:17.053879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:17.389813 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:17.389764 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:17.677106 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:17.677021 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:18.133987 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.133953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:18.134126 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.134016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:18.134126 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.134078 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 10:28:18.134126 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.134099 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:18.134269 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.134134 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:26.134120663 +0000 UTC m=+73.594655386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : configmap "kube-rbac-proxy" not found Jul 07 10:28:18.134269 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.134159 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:26.134146017 +0000 UTC m=+73.594680741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:18.326607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.326572 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:18.711412 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.711369 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:18.739526 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.739499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:28:18.739667 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.739550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:28:18.742546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.742526 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 10:28:18.742620 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.742558 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 10:28:18.750773 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.750751 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 10:28:18.750881 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:18.750868 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs podName:e3e80296-decf-49a0-a5c9-71630ad65571 nodeName:}" failed. No retries permitted until 2026-07-07 10:29:22.750844944 +0000 UTC m=+130.211379684 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs") pod "network-metrics-daemon-hgc7q" (UID: "e3e80296-decf-49a0-a5c9-71630ad65571") : secret "metrics-daemon-secret" not found Jul 07 10:28:18.753038 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.753021 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2087d4e-9d80-4d94-b537-077d2901c6fa-original-pull-secret\") pod \"global-pull-secret-syncer-h9wvp\" (UID: \"a2087d4e-9d80-4d94-b537-077d2901c6fa\") " pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:28:18.891905 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.891866 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wvp" Jul 07 10:28:18.921280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.921244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:18.941624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.941505 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:28:18.944455 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.944428 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 10:28:18.955473 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.955452 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 10:28:18.964978 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:18.964910 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22qh6\" (UniqueName: \"kubernetes.io/projected/27ff8724-8b54-4013-99a7-f3c09dac2063-kube-api-access-22qh6\") pod \"network-check-target-sdbtk\" (UID: \"27ff8724-8b54-4013-99a7-f3c09dac2063\") " pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:28:19.031721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.031691 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9wvp"] Jul 07 10:28:19.035112 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:19.035085 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2087d4e_9d80_4d94_b537_077d2901c6fa.slice/crio-7a12251c2e63094c49798febff8b0d580020a19ad394e69f87a289c610aeafae WatchSource:0}: Error finding container 7a12251c2e63094c49798febff8b0d580020a19ad394e69f87a289c610aeafae: Status 404 returned error can't find the container with id 7a12251c2e63094c49798febff8b0d580020a19ad394e69f87a289c610aeafae Jul 07 10:28:19.169163 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.169125 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:19.201235 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.201215 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdqsn\"" Jul 07 10:28:19.207760 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.207743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:28:19.319046 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.319018 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:19.334057 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.334031 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdbtk"] Jul 07 10:28:19.337105 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:19.337080 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ff8724_8b54_4013_99a7_f3c09dac2063.slice/crio-0cfdcf810157bdf9620979c06f9861c2d3677fc762e9db19b89705f4080631ad WatchSource:0}: Error finding container 0cfdcf810157bdf9620979c06f9861c2d3677fc762e9db19b89705f4080631ad: Status 404 returned error can't find the container with id 0cfdcf810157bdf9620979c06f9861c2d3677fc762e9db19b89705f4080631ad Jul 07 10:28:19.367878 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.367848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdbtk" event={"ID":"27ff8724-8b54-4013-99a7-f3c09dac2063","Type":"ContainerStarted","Data":"0cfdcf810157bdf9620979c06f9861c2d3677fc762e9db19b89705f4080631ad"} Jul 07 10:28:19.368923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.368903 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9wvp" event={"ID":"a2087d4e-9d80-4d94-b537-077d2901c6fa","Type":"ContainerStarted","Data":"7a12251c2e63094c49798febff8b0d580020a19ad394e69f87a289c610aeafae"} Jul 07 10:28:19.967456 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:19.967420 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:20.070288 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.070239 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:20.317458 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.317287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:20.536326 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.536192 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:20.858901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.858860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:20.861748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.861721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b127bf4a-b560-4b09-8f9a-21fe84bc3dfe-metrics-tls\") pod \"dns-default-7ltr2\" (UID: \"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe\") " pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:20.944433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.944398 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:20.959408 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.959369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:20.959408 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:20.959436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:20.959767 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:20.959556 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:36.959532667 +0000 UTC m=+84.420067394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : configmap references non-existent config key: service-ca.crt Jul 07 10:28:20.959767 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:20.959585 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 10:28:20.959767 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:20.959636 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs podName:438503da-c32e-49be-9296-4c1beb56f484 nodeName:}" failed. No retries permitted until 2026-07-07 10:28:36.959619771 +0000 UTC m=+84.420154501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs") pod "router-default-68d68d4cdf-ghwnt" (UID: "438503da-c32e-49be-9296-4c1beb56f484") : secret "router-metrics-certs-default" not found Jul 07 10:28:21.037995 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.037967 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq5jm\"" Jul 07 10:28:21.045558 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.045497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:21.047940 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.047912 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:21.144934 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.144853 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:21.210011 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.209923 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7ltr2"] Jul 07 10:28:21.219366 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:21.214648 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb127bf4a_b560_4b09_8f9a_21fe84bc3dfe.slice/crio-22d92c5bd083e7fcfa1c29e51ee2e0c6f5a34fc77a20b4fc226eabf3a8efbc24 WatchSource:0}: Error finding container 22d92c5bd083e7fcfa1c29e51ee2e0c6f5a34fc77a20b4fc226eabf3a8efbc24: Status 404 returned error can't find the container with id 22d92c5bd083e7fcfa1c29e51ee2e0c6f5a34fc77a20b4fc226eabf3a8efbc24 Jul 07 10:28:21.375784 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.375746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ltr2" event={"ID":"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe","Type":"ContainerStarted","Data":"22d92c5bd083e7fcfa1c29e51ee2e0c6f5a34fc77a20b4fc226eabf3a8efbc24"} Jul 07 10:28:21.515577 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:21.515543 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:22.150339 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:22.150304 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:22.584977 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:22.584868 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:22.674849 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:22.674774 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:22.775407 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:22.775294 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:23.121463 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:23.121414 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:23.573193 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:23.573034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:23.847306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:23.847227 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:23.947959 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:23.947925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:24.115212 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.115128 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:24.309433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.309340 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:24.387253 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.387216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdbtk" event={"ID":"27ff8724-8b54-4013-99a7-f3c09dac2063","Type":"ContainerStarted","Data":"116d07aa6a2169ad520e639d0dca4f62287dcf44ee4d9f2b9d028cec75f73f9c"} Jul 07 10:28:24.387457 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.387317 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:28:24.388844 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.388788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9wvp" event={"ID":"a2087d4e-9d80-4d94-b537-077d2901c6fa","Type":"ContainerStarted","Data":"19d540c35dd463664dfd33a971b9207ed9b3827a9378a5f7bc35fa2f05d00dd7"} Jul 07 10:28:24.406774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.406705 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sdbtk" podStartSLOduration=66.553687615 podStartE2EDuration="1m11.406685746s" totalImagesPullingTime="4.852998131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:24.406247231 +0000 UTC m=+71.866781995" watchObservedRunningTime="2026-07-07 10:28:24.406685746 +0000 UTC m=+71.867220492" Jul 07 10:28:24.707770 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:24.707675 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:25.268531 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:25.268491 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:25.368182 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:25.368148 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:25.913404 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:25.913370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:26.129963 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.129929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:26.205641 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.205558 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:26.205641 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.205595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:26.205834 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:26.205708 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 10:28:26.205834 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:28:26.205768 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls podName:5d34c59e-c9d0-41af-88ea-fd845c1689de nodeName:}" failed. No retries permitted until 2026-07-07 10:28:42.20575168 +0000 UTC m=+89.666286403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8qswp" (UID: "5d34c59e-c9d0-41af-88ea-fd845c1689de") : secret "insights-runtime-extractor-tls" not found Jul 07 10:28:26.206081 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.206065 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5d34c59e-c9d0-41af-88ea-fd845c1689de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:26.349016 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.348980 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:26.397648 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.397621 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ltr2" event={"ID":"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe","Type":"ContainerStarted","Data":"b0c096952cfa88045e4cde33e6d1365ca8ede03ab22d9fa7354b0c59985569ab"} Jul 07 10:28:26.397791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.397700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ltr2" event={"ID":"b127bf4a-b560-4b09-8f9a-21fe84bc3dfe","Type":"ContainerStarted","Data":"2e34e266f82835332e765634469e948c2a99b42e15a2a457083cf0f537cecf8c"} Jul 07 10:28:26.397791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.397766 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:26.426706 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.426633 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h9wvp" podStartSLOduration=68.268710605 podStartE2EDuration="1m13.426617786s" totalImagesPullingTime="5.157907181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:24.422962949 +0000 UTC m=+71.883497708" watchObservedRunningTime="2026-07-07 10:28:26.426617786 +0000 UTC m=+73.887152531" Jul 07 10:28:26.426834 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.426787 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7ltr2" podStartSLOduration=34.077509625 podStartE2EDuration="38.426781668s" totalImagesPullingTime="4.349272043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:26.424658023 +0000 UTC m=+73.885192768" watchObservedRunningTime="2026-07-07 10:28:26.426781668 +0000 UTC m=+73.887316473" Jul 07 10:28:26.537049 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:26.536970 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:27.116051 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:27.116015 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:27.319657 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:27.319619 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:28.464910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:28.464869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:29.559494 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:29.559464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:29.661093 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:29.661054 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:29.769860 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:29.769822 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:29.869896 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:29.869865 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.042210 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.042177 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.141883 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.141779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.239079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.239045 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.316767 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.316728 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.468824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.468718 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.551723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.551691 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:30.648859 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:30.648820 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:31.042821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:31.042777 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:31.161454 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:31.161413 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:31.271540 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:31.271508 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:32.087098 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:32.087057 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:32.165090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:32.165051 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:34.347094 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.346953 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-g2nwq"] Jul 07 10:28:34.348995 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.348977 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:34.358512 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.358495 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-svjq8\"" Jul 07 10:28:34.358621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.358606 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 10:28:34.359773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.359758 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 10:28:34.383926 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.383895 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-g2nwq"] Jul 07 10:28:34.465912 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.465873 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d8d4559f6-k4fxt"] Jul 07 10:28:34.467918 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.467890 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.471666 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.471638 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9kt2\" (UniqueName: \"kubernetes.io/projected/eca3c9be-97c5-4abe-b650-7bf5384ffed1-kube-api-access-z9kt2\") pod \"downloads-685bfb5f64-g2nwq\" (UID: \"eca3c9be-97c5-4abe-b650-7bf5384ffed1\") " pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:34.472052 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.472035 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 10:28:34.472142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.472058 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 10:28:34.477001 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.476979 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:34.483406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.483384 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 10:28:34.484019 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.484002 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tvxzc\"" Jul 07 10:28:34.484920 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.484898 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 10:28:34.520966 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.520940 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d8d4559f6-k4fxt"] Jul 07 10:28:34.573170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-installation-pull-secrets\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-ca-trust-extracted\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573260 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9kt2\" (UniqueName: \"kubernetes.io/projected/eca3c9be-97c5-4abe-b650-7bf5384ffed1-kube-api-access-z9kt2\") pod \"downloads-685bfb5f64-g2nwq\" (UID: \"eca3c9be-97c5-4abe-b650-7bf5384ffed1\") " pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:34.573357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-certificates\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-bound-sa-token\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghlj2\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-kube-api-access-ghlj2\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-image-registry-private-configuration\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573478 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-tls\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.573546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.573529 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-trusted-ca\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.598678 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.598599 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9kt2\" (UniqueName: \"kubernetes.io/projected/eca3c9be-97c5-4abe-b650-7bf5384ffed1-kube-api-access-z9kt2\") pod \"downloads-685bfb5f64-g2nwq\" (UID: \"eca3c9be-97c5-4abe-b650-7bf5384ffed1\") " pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:34.657512 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.657474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:34.666180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.666151 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:34.673989 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.673962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-certificates\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-bound-sa-token\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghlj2\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-kube-api-access-ghlj2\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-image-registry-private-configuration\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-tls\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-trusted-ca\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-installation-pull-secrets\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-ca-trust-extracted\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.674776 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674731 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-ca-trust-extracted\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.675014 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.674987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-certificates\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.675582 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.675555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-trusted-ca\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.677127 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.677098 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-installation-pull-secrets\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.677205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.677122 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-image-registry-private-configuration\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.677251 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.677227 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-registry-tls\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.699334 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.699300 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-bound-sa-token\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.702942 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.702911 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghlj2\" (UniqueName: \"kubernetes.io/projected/8b42335c-84e9-4349-9ad9-e9ba7fc41e00-kube-api-access-ghlj2\") pod \"image-registry-5d8d4559f6-k4fxt\" (UID: \"8b42335c-84e9-4349-9ad9-e9ba7fc41e00\") " pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.776217 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.776184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:34.815908 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.815861 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:34.816059 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.815978 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-g2nwq"] Jul 07 10:28:34.818590 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:34.818558 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca3c9be_97c5_4abe_b650_7bf5384ffed1.slice/crio-01db2935330efa228ff8a3bb9395d4f1172bb03fc5102059a30bdadd14057d79 WatchSource:0}: Error finding container 01db2935330efa228ff8a3bb9395d4f1172bb03fc5102059a30bdadd14057d79: Status 404 returned error can't find the container with id 01db2935330efa228ff8a3bb9395d4f1172bb03fc5102059a30bdadd14057d79 Jul 07 10:28:34.930136 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:34.930102 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d8d4559f6-k4fxt"] Jul 07 10:28:34.938497 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:34.938470 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b42335c_84e9_4349_9ad9_e9ba7fc41e00.slice/crio-e9753133f1bae392c432a661f59536e374638b4b36cdbaa4be3f97550ba4db10 WatchSource:0}: Error finding container e9753133f1bae392c432a661f59536e374638b4b36cdbaa4be3f97550ba4db10: Status 404 returned error can't find the container with id e9753133f1bae392c432a661f59536e374638b4b36cdbaa4be3f97550ba4db10 Jul 07 10:28:35.026702 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.026671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:35.427624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.427586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" event={"ID":"8b42335c-84e9-4349-9ad9-e9ba7fc41e00","Type":"ContainerStarted","Data":"6cac1b0fb7a966c23e684af9484c8990bc3637783510e1140f3b85f4a254360d"} Jul 07 10:28:35.428099 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.427648 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:35.428099 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.427669 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" event={"ID":"8b42335c-84e9-4349-9ad9-e9ba7fc41e00","Type":"ContainerStarted","Data":"e9753133f1bae392c432a661f59536e374638b4b36cdbaa4be3f97550ba4db10"} Jul 07 10:28:35.429266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.429242 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-g2nwq" event={"ID":"eca3c9be-97c5-4abe-b650-7bf5384ffed1","Type":"ContainerStarted","Data":"01db2935330efa228ff8a3bb9395d4f1172bb03fc5102059a30bdadd14057d79"} Jul 07 10:28:35.472054 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:35.472002 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" podStartSLOduration=1.4719844069999999 podStartE2EDuration="1.471984407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:35.471867046 +0000 UTC m=+82.932401809" watchObservedRunningTime="2026-07-07 10:28:35.471984407 +0000 UTC m=+82.932519174" Jul 07 10:28:36.402930 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:36.402900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7ltr2" Jul 07 10:28:36.992724 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:36.992691 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:36.993185 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:36.992790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:36.993433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:36.993405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438503da-c32e-49be-9296-4c1beb56f484-service-ca-bundle\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:36.995722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:36.995698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/438503da-c32e-49be-9296-4c1beb56f484-metrics-certs\") pod \"router-default-68d68d4cdf-ghwnt\" (UID: \"438503da-c32e-49be-9296-4c1beb56f484\") " pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:37.187654 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.187621 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tzdlj\"" Jul 07 10:28:37.195028 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.194998 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:37.352486 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.352404 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68d68d4cdf-ghwnt"] Jul 07 10:28:37.359075 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:37.359045 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438503da_c32e_49be_9296_4c1beb56f484.slice/crio-128b2dbc09e39a6948ba9195d8b54966dd6d510c3fff4515bc6c257325c6fc6a WatchSource:0}: Error finding container 128b2dbc09e39a6948ba9195d8b54966dd6d510c3fff4515bc6c257325c6fc6a: Status 404 returned error can't find the container with id 128b2dbc09e39a6948ba9195d8b54966dd6d510c3fff4515bc6c257325c6fc6a Jul 07 10:28:37.440841 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.440784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" event={"ID":"438503da-c32e-49be-9296-4c1beb56f484","Type":"ContainerStarted","Data":"f462d05212a8d1c071bb4c6a37e01c16922208f9fb6330f3c105b52f8216a56e"} Jul 07 10:28:37.440997 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.440874 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" event={"ID":"438503da-c32e-49be-9296-4c1beb56f484","Type":"ContainerStarted","Data":"128b2dbc09e39a6948ba9195d8b54966dd6d510c3fff4515bc6c257325c6fc6a"} Jul 07 10:28:37.474281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.474230 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" podStartSLOduration=33.474215217 podStartE2EDuration="33.474215217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:37.473502005 +0000 UTC m=+84.934036770" watchObservedRunningTime="2026-07-07 10:28:37.474215217 +0000 UTC m=+84.934749962" Jul 07 10:28:37.539882 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.539776 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:37.662354 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:37.662316 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:38.195425 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.195381 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:38.198263 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.198233 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:38.443944 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.443917 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:38.445285 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.445260 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68d68d4cdf-ghwnt" Jul 07 10:28:38.851817 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.851719 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:38.952716 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:38.952682 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:39.450783 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:39.450746 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:39.552762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:39.552729 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:39.983840 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:39.983732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:40.083178 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:40.083141 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:40.169567 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:40.169530 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:40.253207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:40.253051 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:42.236066 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:42.236029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:42.238950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:42.238913 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5d34c59e-c9d0-41af-88ea-fd845c1689de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8qswp\" (UID: \"5d34c59e-c9d0-41af-88ea-fd845c1689de\") " pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:42.479659 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:42.479631 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wt2hz\"" Jul 07 10:28:42.486582 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:42.486519 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8qswp" Jul 07 10:28:42.674924 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:42.674892 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8qswp"] Jul 07 10:28:42.679348 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:42.679309 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d34c59e_c9d0_41af_88ea_fd845c1689de.slice/crio-693e7a664c5398d5c6c701afc9257ca6bd1f17a05eecc7724254bdc44048cbbf WatchSource:0}: Error finding container 693e7a664c5398d5c6c701afc9257ca6bd1f17a05eecc7724254bdc44048cbbf: Status 404 returned error can't find the container with id 693e7a664c5398d5c6c701afc9257ca6bd1f17a05eecc7724254bdc44048cbbf Jul 07 10:28:43.199237 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.199197 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.317350 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.317307 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.464914 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.464823 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8qswp" event={"ID":"5d34c59e-c9d0-41af-88ea-fd845c1689de","Type":"ContainerStarted","Data":"693e7a664c5398d5c6c701afc9257ca6bd1f17a05eecc7724254bdc44048cbbf"} Jul 07 10:28:43.472448 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.472421 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.574898 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.574860 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.778701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.778527 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.890720 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.890685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:43.987619 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:43.987572 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:45.584517 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.584476 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:45.674106 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.674075 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:45.875343 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.875252 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:28:45.881750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.881719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.884133 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.884054 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:45.884721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.884701 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 10:28:45.884721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.884717 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 10:28:45.884898 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.884825 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d4kbr\"" Jul 07 10:28:45.885976 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.885825 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 10:28:45.885976 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.885851 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 10:28:45.886193 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.886078 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 10:28:45.895052 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.895029 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:28:45.895203 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.895182 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 10:28:45.966729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.966688 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:45.969588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87czv\" (UniqueName: \"kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969579 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969619 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:45.969925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:45.969677 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070384 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87czv\" (UniqueName: \"kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070786 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070677 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.070786 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.070723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.071396 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.071363 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.071508 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.071367 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.071508 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.071403 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.071691 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.071637 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.073623 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.073601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.073791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.073772 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.080880 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.080855 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87czv\" (UniqueName: \"kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv\") pod \"console-f4c45c5f6-bq6xv\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.192543 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.192464 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:28:46.779727 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:46.779698 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:28:46.782210 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:46.782180 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c69b28d_bd70_4e50_ad8a_c4cdb450bce9.slice/crio-6a3bfe95a36a637ffddfa39e65eb7de65df5a32140f1a1324dbc081ee19c7ee6 WatchSource:0}: Error finding container 6a3bfe95a36a637ffddfa39e65eb7de65df5a32140f1a1324dbc081ee19c7ee6: Status 404 returned error can't find the container with id 6a3bfe95a36a637ffddfa39e65eb7de65df5a32140f1a1324dbc081ee19c7ee6 Jul 07 10:28:47.482276 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:47.482233 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c45c5f6-bq6xv" event={"ID":"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9","Type":"ContainerStarted","Data":"6a3bfe95a36a637ffddfa39e65eb7de65df5a32140f1a1324dbc081ee19c7ee6"} Jul 07 10:28:47.484174 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:47.484147 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8qswp" event={"ID":"5d34c59e-c9d0-41af-88ea-fd845c1689de","Type":"ContainerStarted","Data":"15b894b79978bd505e7d6288bf09438087d484f4f2a708414766768df61745dd"} Jul 07 10:28:48.007156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:48.007120 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:48.092684 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:48.092610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:48.156310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:48.156196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:50.438192 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.438106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:50.725170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.725086 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:50.811556 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.811304 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4"] Jul 07 10:28:50.815097 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.815074 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.820450 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.820297 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 10:28:50.820450 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.820312 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 10:28:50.820644 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.820577 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 10:28:50.821947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.821914 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 10:28:50.828206 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.828184 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 10:28:50.829938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.829919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cfgtv\"" Jul 07 10:28:50.841553 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.841531 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 10:28:50.850080 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.850055 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4"] Jul 07 10:28:50.854226 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.854203 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mbqq9"] Jul 07 10:28:50.857669 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.857650 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:50.863774 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.863749 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 10:28:50.863927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.863757 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 10:28:50.863927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.863762 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 10:28:50.863927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.863876 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lfkb5\"" Jul 07 10:28:50.892879 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.892845 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:50.911332 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7k2q\" (UniqueName: \"kubernetes.io/projected/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-api-access-j7k2q\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.911495 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.911495 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c68b91c-e816-4609-8162-3fc148c2fb4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.911589 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.911636 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911580 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:50.911636 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:50.911615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012477 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-root\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.012477 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012694 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012543 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012694 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-metrics-client-ca\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.012694 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012618 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012694 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012658 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-textfile\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf6fp\" (UniqueName: \"kubernetes.io/projected/be115682-2260-46a2-a57e-0e23166e00f7-kube-api-access-bf6fp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012733 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7k2q\" (UniqueName: \"kubernetes.io/projected/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-api-access-j7k2q\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c68b91c-e816-4609-8162-3fc148c2fb4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012873 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-sys\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.012932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-wtmp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.013225 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.013225 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-tls\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.013225 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.012982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.013410 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.013265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.013410 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.013299 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c68b91c-e816-4609-8162-3fc148c2fb4e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.013410 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.013364 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c68b91c-e816-4609-8162-3fc148c2fb4e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.015987 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.015964 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.016561 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.016540 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.029174 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.029156 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7k2q\" (UniqueName: \"kubernetes.io/projected/2c68b91c-e816-4609-8162-3fc148c2fb4e-kube-api-access-j7k2q\") pod \"kube-state-metrics-7c9dbb8b7d-75ct4\" (UID: \"2c68b91c-e816-4609-8162-3fc148c2fb4e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.060969 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.060924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:51.114189 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-metrics-client-ca\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-textfile\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf6fp\" (UniqueName: \"kubernetes.io/projected/be115682-2260-46a2-a57e-0e23166e00f7-kube-api-access-bf6fp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114285 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-sys\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-wtmp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114598 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114372 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-tls\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114598 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114396 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114598 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114426 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-root\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114598 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-sys\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114598 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-root\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114842 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-wtmp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114842 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114676 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-textfile\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.114911 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114867 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-metrics-client-ca\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.115006 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.114986 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.117177 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.117154 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.117283 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.117260 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be115682-2260-46a2-a57e-0e23166e00f7-node-exporter-tls\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.125570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.125534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" Jul 07 10:28:51.125958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.125925 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf6fp\" (UniqueName: \"kubernetes.io/projected/be115682-2260-46a2-a57e-0e23166e00f7-kube-api-access-bf6fp\") pod \"node-exporter-mbqq9\" (UID: \"be115682-2260-46a2-a57e-0e23166e00f7\") " pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.169507 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.169475 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mbqq9" Jul 07 10:28:51.199707 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.199672 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:51.326538 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.326381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:51.512923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.512875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:51.890453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:51.890359 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:52.216587 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:52.216503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:52.434300 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:52.434263 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:52.623185 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:52.623156 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:53.132490 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:53.132459 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:53.283517 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:53.283480 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:54.368403 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:54.368358 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe115682_2260_46a2_a57e_0e23166e00f7.slice/crio-3bea5c889229e8f5f0119ed2d2823b32abaefba2b7696ea47f63aa82c7fa6ed8 WatchSource:0}: Error finding container 3bea5c889229e8f5f0119ed2d2823b32abaefba2b7696ea47f63aa82c7fa6ed8: Status 404 returned error can't find the container with id 3bea5c889229e8f5f0119ed2d2823b32abaefba2b7696ea47f63aa82c7fa6ed8 Jul 07 10:28:54.490474 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:54.490437 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:54.511760 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:54.511723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbqq9" event={"ID":"be115682-2260-46a2-a57e-0e23166e00f7","Type":"ContainerStarted","Data":"3bea5c889229e8f5f0119ed2d2823b32abaefba2b7696ea47f63aa82c7fa6ed8"} Jul 07 10:28:54.546704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:54.546680 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4"] Jul 07 10:28:54.669576 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:54.669541 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:54.741072 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:28:54.741022 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c68b91c_e816_4609_8162_3fc148c2fb4e.slice/crio-249e4d39acc7f604b8a88d8321cc23ed04651a4a0a2f818c77f51fa58102fdf6 WatchSource:0}: Error finding container 249e4d39acc7f604b8a88d8321cc23ed04651a4a0a2f818c77f51fa58102fdf6: Status 404 returned error can't find the container with id 249e4d39acc7f604b8a88d8321cc23ed04651a4a0a2f818c77f51fa58102fdf6 Jul 07 10:28:54.843236 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:54.843207 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:55.160608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.160569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:55.396704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.394941 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sdbtk" Jul 07 10:28:55.507444 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.507407 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b9f9468d6-8g9hd"] Jul 07 10:28:55.524530 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.524499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:55.531199 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531172 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:55.531381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-g2nwq" event={"ID":"eca3c9be-97c5-4abe-b650-7bf5384ffed1","Type":"ContainerStarted","Data":"a01bda6067568efdd0db61004876e9de9e1f6d94404f741f515e6c191dc78c8a"} Jul 07 10:28:55.531381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531264 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-g2nwq" Jul 07 10:28:55.531381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531286 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" event={"ID":"2c68b91c-e816-4609-8162-3fc148c2fb4e","Type":"ContainerStarted","Data":"249e4d39acc7f604b8a88d8321cc23ed04651a4a0a2f818c77f51fa58102fdf6"} Jul 07 10:28:55.531381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531312 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8qswp" event={"ID":"5d34c59e-c9d0-41af-88ea-fd845c1689de","Type":"ContainerStarted","Data":"6c5fbeb74b964e8f21d1577b0ed1bfdcd825d427a0aef1ba640ee5e6fc073a0b"} Jul 07 10:28:55.531381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.531364 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.536450 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.536428 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 10:28:55.536706 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.536689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 10:28:55.537022 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.536906 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 10:28:55.537611 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.537592 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-96ckz\"" Jul 07 10:28:55.537861 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.537843 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 10:28:55.538063 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.538047 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3vkq0efpjf33a\"" Jul 07 10:28:55.556868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.551607 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b9f9468d6-8g9hd"] Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557305 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-metrics-server-audit-profiles\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557358 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-client-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-client-certs\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfa81a65-6f6e-450c-9567-501f3d844a85-audit-log\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqdtf\" (UniqueName: \"kubernetes.io/projected/cfa81a65-6f6e-450c-9567-501f3d844a85-kube-api-access-vqdtf\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.558125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.557651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-tls\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.570449 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.570391 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-g2nwq" podStartSLOduration=1.560503639 podStartE2EDuration="21.570372603s" totalImagesPullingTime="20.009868964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:55.569765564 +0000 UTC m=+103.030300332" watchObservedRunningTime="2026-07-07 10:28:55.570372603 +0000 UTC m=+103.030907348" Jul 07 10:28:55.658463 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658384 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-metrics-server-audit-profiles\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658463 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-client-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-client-certs\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfa81a65-6f6e-450c-9567-501f3d844a85-audit-log\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqdtf\" (UniqueName: \"kubernetes.io/projected/cfa81a65-6f6e-450c-9567-501f3d844a85-kube-api-access-vqdtf\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.658704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.658627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-tls\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.659639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.659466 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-metrics-server-audit-profiles\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.659831 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.659790 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cfa81a65-6f6e-450c-9567-501f3d844a85-audit-log\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.660991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.660824 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cfa81a65-6f6e-450c-9567-501f3d844a85-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.662022 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.661971 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-tls\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.663251 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.662753 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-client-ca-bundle\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.663251 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.663208 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cfa81a65-6f6e-450c-9567-501f3d844a85-secret-metrics-server-client-certs\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.690991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.690953 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqdtf\" (UniqueName: \"kubernetes.io/projected/cfa81a65-6f6e-450c-9567-501f3d844a85-kube-api-access-vqdtf\") pod \"metrics-server-5b9f9468d6-8g9hd\" (UID: \"cfa81a65-6f6e-450c-9567-501f3d844a85\") " pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:55.768199 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.768157 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:55.856164 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:55.856126 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:28:56.179832 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.178990 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b9f9468d6-8g9hd"] Jul 07 10:28:56.360415 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.360337 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:56.439773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.439744 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d8d4559f6-k4fxt" Jul 07 10:28:56.479360 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.479322 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:56.533073 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.533035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" event={"ID":"cfa81a65-6f6e-450c-9567-501f3d844a85","Type":"ContainerStarted","Data":"04eac7bd550c40a02228bc60036ab4c4843eeaf0f9c843eab0fba55b5af1872b"} Jul 07 10:28:56.535407 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.535372 2581 generic.go:356] "Generic (PLEG): container finished" podID="be115682-2260-46a2-a57e-0e23166e00f7" containerID="0c0f07eabb8af11017a6b819735ce8bff83ce9232642a857252294902345564a" exitCode=0 Jul 07 10:28:56.536336 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.536311 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbqq9" event={"ID":"be115682-2260-46a2-a57e-0e23166e00f7","Type":"ContainerDied","Data":"0c0f07eabb8af11017a6b819735ce8bff83ce9232642a857252294902345564a"} Jul 07 10:28:56.557050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.557021 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:28:56.691018 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.690973 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:56.792935 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.792899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:56.892389 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:56.892356 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:57.544529 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:57.544488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbqq9" event={"ID":"be115682-2260-46a2-a57e-0e23166e00f7","Type":"ContainerStarted","Data":"e0ef87311288ec6651e4ef567cf30c7e01f5e1f4db7f5002042298ab1e65708d"} Jul 07 10:28:57.545026 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:57.544559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mbqq9" event={"ID":"be115682-2260-46a2-a57e-0e23166e00f7","Type":"ContainerStarted","Data":"c9db0c733e403bc60a1e6bdc043b4c9058764777f647f00e83349fcd1203749a"} Jul 07 10:28:58.101550 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:58.101504 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:59.026893 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:59.026767 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:59.124639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:59.124548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:59.575714 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:59.575548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:59.687524 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:59.687477 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:28:59.796477 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:28:59.796433 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:02.567928 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.567463 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c45c5f6-bq6xv" event={"ID":"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9","Type":"ContainerStarted","Data":"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1"} Jul 07 10:29:02.569258 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.569227 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" event={"ID":"cfa81a65-6f6e-450c-9567-501f3d844a85","Type":"ContainerStarted","Data":"dab5c986c70891b40fc004d8f1375e4b641a77ba49606bc9b5d123a386368c36"} Jul 07 10:29:02.573693 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.573171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" event={"ID":"2c68b91c-e816-4609-8162-3fc148c2fb4e","Type":"ContainerStarted","Data":"c7035f29fa435a2d8aa2587dd1f914955b2d2af8b597d0d0178f7fc603dcf706"} Jul 07 10:29:02.573693 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.573223 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" event={"ID":"2c68b91c-e816-4609-8162-3fc148c2fb4e","Type":"ContainerStarted","Data":"4df42ce8f30e861b2c7f3c5a432df8343865c3a9ea25d4df376c253f99bd182a"} Jul 07 10:29:02.573693 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.573253 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" event={"ID":"2c68b91c-e816-4609-8162-3fc148c2fb4e","Type":"ContainerStarted","Data":"89f5fd6864448aaf87478d90b252ed3e753b888b15fc4f55adaac27ed8071709"} Jul 07 10:29:02.575601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.575576 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8qswp" event={"ID":"5d34c59e-c9d0-41af-88ea-fd845c1689de","Type":"ContainerStarted","Data":"ba4ad58907a88cdaf9a3aab530dc3a944502efeff64e3ac6128a230ddc6ecf5a"} Jul 07 10:29:02.606253 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.606190 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f4c45c5f6-bq6xv" podStartSLOduration=2.470701729 podStartE2EDuration="17.606168106s" totalImagesPullingTime="15.135466377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:29:02.604519759 +0000 UTC m=+110.065054523" watchObservedRunningTime="2026-07-07 10:29:02.606168106 +0000 UTC m=+110.066702853" Jul 07 10:29:02.606690 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.606651 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mbqq9" podStartSLOduration=10.876693107 podStartE2EDuration="12.606639962s" totalImagesPullingTime="1.592193173s" totalInitContainerRuntime="137.753682ms" isStatefulPod=true podCreationTimestamp="2026-07-07 10:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:28:57.597823106 +0000 UTC m=+105.058357877" watchObservedRunningTime="2026-07-07 10:29:02.606639962 +0000 UTC m=+110.067174710" Jul 07 10:29:02.666594 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.666532 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-75ct4" podStartSLOduration=5.523722673 podStartE2EDuration="12.666507385s" totalImagesPullingTime="7.142784712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:29:02.665251185 +0000 UTC m=+110.125785996" watchObservedRunningTime="2026-07-07 10:29:02.666507385 +0000 UTC m=+110.127042172" Jul 07 10:29:02.688675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:02.687480 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8qswp" podStartSLOduration=33.637557549 podStartE2EDuration="52.687456067s" totalImagesPullingTime="19.049898518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 10:28:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:29:02.686356962 +0000 UTC m=+110.146891730" watchObservedRunningTime="2026-07-07 10:29:02.687456067 +0000 UTC m=+110.147990813" Jul 07 10:29:04.675144 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:04.675095 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:04.776219 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:04.776179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:04.877110 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:04.877074 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:06.193188 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:06.193156 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:29:07.596171 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:07.596126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:07.712785 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:07.712749 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:08.146161 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:08.146119 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:08.212394 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:08.212346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:08.838638 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:08.838604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:08.942503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:08.942459 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:15.433209 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.433173 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:15.503790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.503744 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:15.620880 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.620844 2581 generic.go:356] "Generic (PLEG): container finished" podID="588b9ad5-e0c5-4fbf-9771-32305e6445c6" containerID="8d502af8c01dc82a43c7e59cd602a1c550e5749e3f839c439c58f8b2d8ce4659" exitCode=0 Jul 07 10:29:15.621048 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.620905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" event={"ID":"588b9ad5-e0c5-4fbf-9771-32305e6445c6","Type":"ContainerDied","Data":"8d502af8c01dc82a43c7e59cd602a1c550e5749e3f839c439c58f8b2d8ce4659"} Jul 07 10:29:15.621218 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.621203 2581 scope.go:117] "RemoveContainer" containerID="8d502af8c01dc82a43c7e59cd602a1c550e5749e3f839c439c58f8b2d8ce4659" Jul 07 10:29:15.640092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.640048 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" podStartSLOduration=14.905353286 podStartE2EDuration="20.640030632s" totalImagesPullingTime="5.734677346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:29:02.714194616 +0000 UTC m=+110.174729383" watchObservedRunningTime="2026-07-07 10:29:15.640030632 +0000 UTC m=+123.100565378" Jul 07 10:29:15.856930 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.856884 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:29:15.857125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:15.856949 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:29:16.625890 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:16.625858 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jbgzb" event={"ID":"588b9ad5-e0c5-4fbf-9771-32305e6445c6","Type":"ContainerStarted","Data":"ca58f8e1875a7f2619af5bafd49d561b17ff3964e94f8682beae05ba079dde74"} Jul 07 10:29:22.802392 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:22.802355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:29:22.804875 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:22.804852 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3e80296-decf-49a0-a5c9-71630ad65571-metrics-certs\") pod \"network-metrics-daemon-hgc7q\" (UID: \"e3e80296-decf-49a0-a5c9-71630ad65571\") " pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:29:22.988081 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:22.988046 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-68x57\"" Jul 07 10:29:22.995863 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:22.995844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hgc7q" Jul 07 10:29:23.141153 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:23.141122 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hgc7q"] Jul 07 10:29:23.144354 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:29:23.144328 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e80296_decf_49a0_a5c9_71630ad65571.slice/crio-6dd4904e59a8e5a9ee2571817871fb940e3cb965a0ce3db0b416e10dd7094208 WatchSource:0}: Error finding container 6dd4904e59a8e5a9ee2571817871fb940e3cb965a0ce3db0b416e10dd7094208: Status 404 returned error can't find the container with id 6dd4904e59a8e5a9ee2571817871fb940e3cb965a0ce3db0b416e10dd7094208 Jul 07 10:29:23.652982 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:23.652932 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgc7q" event={"ID":"e3e80296-decf-49a0-a5c9-71630ad65571","Type":"ContainerStarted","Data":"6dd4904e59a8e5a9ee2571817871fb940e3cb965a0ce3db0b416e10dd7094208"} Jul 07 10:29:23.863112 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:23.863072 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:23.962284 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:23.962253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:24.612762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.612719 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:24.658214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.658176 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgc7q" event={"ID":"e3e80296-decf-49a0-a5c9-71630ad65571","Type":"ContainerStarted","Data":"3cf68e3397cf921771ae166a4a8213b5e05d52f0af5df9400505543556696de1"} Jul 07 10:29:24.658351 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.658238 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hgc7q" event={"ID":"e3e80296-decf-49a0-a5c9-71630ad65571","Type":"ContainerStarted","Data":"c2b4c763d1c0496775bec11b29160d610c18c1efb890913ed3fa142cccafaf3d"} Jul 07 10:29:24.680965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.680911 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hgc7q" podStartSLOduration=130.919028385 podStartE2EDuration="2m11.680895925s" totalImagesPullingTime="761.86754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:29:24.678681472 +0000 UTC m=+132.139216229" watchObservedRunningTime="2026-07-07 10:29:24.680895925 +0000 UTC m=+132.141430670" Jul 07 10:29:24.714632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.714605 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:24.846291 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.846259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:24.946448 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:24.946408 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:25.048038 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:25.048006 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:25.662975 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:25.662942 2581 generic.go:356] "Generic (PLEG): container finished" podID="ec72510f-f217-42bf-90a6-05669e0f08fb" containerID="e51148ce28693364d2cda9c5f8cddb9b84a15f15f7ab62b8a9689acab5c8243b" exitCode=0 Jul 07 10:29:25.663163 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:25.663015 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" event={"ID":"ec72510f-f217-42bf-90a6-05669e0f08fb","Type":"ContainerDied","Data":"e51148ce28693364d2cda9c5f8cddb9b84a15f15f7ab62b8a9689acab5c8243b"} Jul 07 10:29:25.663367 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:25.663352 2581 scope.go:117] "RemoveContainer" containerID="e51148ce28693364d2cda9c5f8cddb9b84a15f15f7ab62b8a9689acab5c8243b" Jul 07 10:29:26.668332 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:26.668298 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-tf2f8" event={"ID":"ec72510f-f217-42bf-90a6-05669e0f08fb","Type":"ContainerStarted","Data":"376fe62ecd578685ff22c6e6c840128e44b39b501b0c4b84af9755d18ceb0b9a"} Jul 07 10:29:27.596223 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.596188 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f4c45c5f6-bq6xv" podUID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" containerName="console" containerID="cri-o://b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1" gracePeriod=15 Jul 07 10:29:27.863067 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.862993 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:29:27.945461 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945433 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945618 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945476 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945618 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945504 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945618 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945566 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945725 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945617 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945725 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945660 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.945725 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.945695 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87czv\" (UniqueName: \"kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv\") pod \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\" (UID: \"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9\") " Jul 07 10:29:27.946031 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.946001 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:29:27.946109 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.946052 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:29:27.946109 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.946071 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca" (OuterVolumeSpecName: "service-ca") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:29:27.946210 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.946132 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config" (OuterVolumeSpecName: "console-config") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:29:27.947788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.947767 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:29:27.947967 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.947944 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv" (OuterVolumeSpecName: "kube-api-access-87czv") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "kube-api-access-87czv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:29:27.948051 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:27.948027 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" (UID: "3c69b28d-bd70-4e50-ad8a-c4cdb450bce9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:29:28.046958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046921 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-service-ca\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.046958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046953 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-config\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.046958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046965 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87czv\" (UniqueName: \"kubernetes.io/projected/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-kube-api-access-87czv\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.047196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046973 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-serving-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.047196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046982 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-trusted-ca-bundle\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.047196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.046992 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-console-oauth-config\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.047196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.047001 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9-oauth-serving-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:29:28.678185 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.678143 2581 generic.go:356] "Generic (PLEG): container finished" podID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" containerID="b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1" exitCode=0 Jul 07 10:29:28.678419 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.678227 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c45c5f6-bq6xv" event={"ID":"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9","Type":"ContainerDied","Data":"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1"} Jul 07 10:29:28.678419 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.678326 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f4c45c5f6-bq6xv" event={"ID":"3c69b28d-bd70-4e50-ad8a-c4cdb450bce9","Type":"ContainerDied","Data":"6a3bfe95a36a637ffddfa39e65eb7de65df5a32140f1a1324dbc081ee19c7ee6"} Jul 07 10:29:28.678419 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.678356 2581 scope.go:117] "RemoveContainer" containerID="b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1" Jul 07 10:29:28.678572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.678550 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f4c45c5f6-bq6xv" Jul 07 10:29:28.690280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.690260 2581 scope.go:117] "RemoveContainer" containerID="b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1" Jul 07 10:29:28.690560 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:29:28.690539 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1\": container with ID starting with b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1 not found: ID does not exist" containerID="b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1" Jul 07 10:29:28.690608 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.690568 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1"} err="failed to get container status \"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1\": rpc error: code = NotFound desc = could not find container \"b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1\": container with ID starting with b0e5978e2d8aa7a979b5f33956e3eedeb603f620bcb786d2699e7a40dc11f5e1 not found: ID does not exist" Jul 07 10:29:28.705257 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.705233 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:29:28.707752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:28.707724 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f4c45c5f6-bq6xv"] Jul 07 10:29:29.078349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:29.078255 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" path="/var/lib/kubelet/pods/3c69b28d-bd70-4e50-ad8a-c4cdb450bce9/volumes" Jul 07 10:29:34.115790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:34.115747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:34.201388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:34.201340 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:34.708210 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:34.708177 2581 generic.go:356] "Generic (PLEG): container finished" podID="01eb5e09-8da9-4011-a831-235ee4ba0ce2" containerID="2d414e496568ad9218a080e0890ddedaf6c764106deab7793981b4f2d6518165" exitCode=0 Jul 07 10:29:34.708378 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:34.708248 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" event={"ID":"01eb5e09-8da9-4011-a831-235ee4ba0ce2","Type":"ContainerDied","Data":"2d414e496568ad9218a080e0890ddedaf6c764106deab7793981b4f2d6518165"} Jul 07 10:29:34.708606 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:34.708590 2581 scope.go:117] "RemoveContainer" containerID="2d414e496568ad9218a080e0890ddedaf6c764106deab7793981b4f2d6518165" Jul 07 10:29:35.713065 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:35.713033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-v22dl" event={"ID":"01eb5e09-8da9-4011-a831-235ee4ba0ce2","Type":"ContainerStarted","Data":"7b819db5785782e5e96c3b38d47683cba2ded950b8bb774496cf5a94eaac2acd"} Jul 07 10:29:35.862288 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:35.862261 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:29:35.866000 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:35.865979 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b9f9468d6-8g9hd" Jul 07 10:29:38.853661 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:38.853619 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:38.939515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:38.939484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:29:57.446378 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:29:57.446340 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:10.939756 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:10.939668 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:11.082750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:11.082705 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:19.539235 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:19.539193 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:19.630490 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:19.630448 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:19.832065 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:19.831975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:30.879932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:30.879878 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:30.979092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:30.979060 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:31.056781 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:31.056742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:57.010729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:57.010696 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:30:57.072152 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:30:57.072101 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:26.210280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:26.210236 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:26.284398 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:26.284361 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:29.551632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.551600 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:31:29.552008 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.551995 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" containerName="console" Jul 07 10:31:29.552053 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.552013 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" containerName="console" Jul 07 10:31:29.552086 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.552079 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c69b28d-bd70-4e50-ad8a-c4cdb450bce9" containerName="console" Jul 07 10:31:29.554347 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.554328 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.557366 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.557341 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 10:31:29.557461 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.557354 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 10:31:29.557513 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.557488 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 10:31:29.558522 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.558485 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 10:31:29.558632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.558542 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d4kbr\"" Jul 07 10:31:29.558632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.558545 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 10:31:29.562921 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.562904 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 10:31:29.572177 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.572157 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:31:29.633323 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.633296 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:29.684977 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.684943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.684977 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.684981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77sw\" (UniqueName: \"kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.685232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.685024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.685232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.685044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.685232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.685143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.685232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.685184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.685232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.685224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.697336 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.697305 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:29.786559 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786559 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786630 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.786787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.786701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77sw\" (UniqueName: \"kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.787489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.787463 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.787592 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.787451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.787638 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.787536 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.787638 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.787550 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.789205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.789180 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.789280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.789249 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.796088 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.796066 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77sw\" (UniqueName: \"kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw\") pod \"console-7448f57fc7-nqkww\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.864230 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.864142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:29.999902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:29.999822 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:31:30.002489 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:31:30.002462 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122cc752_8177_4f01_a8e4_1e35ba39034d.slice/crio-177c668ae43b2d9b299432912683294a78f6b05aab40c56e28ec42ae95c89dfb WatchSource:0}: Error finding container 177c668ae43b2d9b299432912683294a78f6b05aab40c56e28ec42ae95c89dfb: Status 404 returned error can't find the container with id 177c668ae43b2d9b299432912683294a78f6b05aab40c56e28ec42ae95c89dfb Jul 07 10:31:30.124552 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:30.124520 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7448f57fc7-nqkww" event={"ID":"122cc752-8177-4f01-a8e4-1e35ba39034d","Type":"ContainerStarted","Data":"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6"} Jul 07 10:31:30.124747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:30.124573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7448f57fc7-nqkww" event={"ID":"122cc752-8177-4f01-a8e4-1e35ba39034d","Type":"ContainerStarted","Data":"177c668ae43b2d9b299432912683294a78f6b05aab40c56e28ec42ae95c89dfb"} Jul 07 10:31:30.145680 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:30.145620 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7448f57fc7-nqkww" podStartSLOduration=1.145605194 podStartE2EDuration="1.145605194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:31:30.143818708 +0000 UTC m=+257.604353469" watchObservedRunningTime="2026-07-07 10:31:30.145605194 +0000 UTC m=+257.606139942" Jul 07 10:31:38.229400 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:38.229313 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:38.287104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:38.287056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:39.865051 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:39.865008 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:39.865439 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:39.865075 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:39.869983 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:39.869958 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:40.162446 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:40.162418 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:31:40.401319 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:40.401262 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:40.500169 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:40.500081 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:53.843406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:53.843366 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:53.978653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:53.978604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:56.333308 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:56.333261 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:31:56.427879 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:31:56.427850 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:12.938678 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:12.938649 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 10:32:16.180578 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:16.180529 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:16.278821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:16.278760 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:40.137139 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:40.137087 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:41.686057 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:41.686017 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:43.464009 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:43.463962 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:44.749200 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:44.749162 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:45.305818 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.305771 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m"] Jul 07 10:32:45.308564 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.308538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.311635 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.311616 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 10:32:45.312228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.312210 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zjh6k\"" Jul 07 10:32:45.312754 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.312732 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:32:45.324760 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.324740 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m"] Jul 07 10:32:45.498077 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.498044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.498254 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.498110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczdf\" (UniqueName: \"kubernetes.io/projected/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-kube-api-access-rczdf\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.599171 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.599080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczdf\" (UniqueName: \"kubernetes.io/projected/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-kube-api-access-rczdf\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.599171 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.599134 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.599502 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.599485 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.609273 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.609246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczdf\" (UniqueName: \"kubernetes.io/projected/f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c-kube-api-access-rczdf\") pod \"cert-manager-operator-controller-manager-55c444bf86-pgm7m\" (UID: \"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.618051 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.618027 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" Jul 07 10:32:45.751850 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.751828 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m"] Jul 07 10:32:45.754815 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:32:45.754771 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a3c2e5_f1c7_4ae6_8d22_4463ea7ec16c.slice/crio-5926ef64eec9f324c9814ba939d48cf65e8323b322d589b0caf55baead0b81d9 WatchSource:0}: Error finding container 5926ef64eec9f324c9814ba939d48cf65e8323b322d589b0caf55baead0b81d9: Status 404 returned error can't find the container with id 5926ef64eec9f324c9814ba939d48cf65e8323b322d589b0caf55baead0b81d9 Jul 07 10:32:45.757407 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:45.757391 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 10:32:46.400851 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:46.400819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" event={"ID":"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c","Type":"ContainerStarted","Data":"5926ef64eec9f324c9814ba939d48cf65e8323b322d589b0caf55baead0b81d9"} Jul 07 10:32:47.258762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:47.258720 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:47.355457 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:47.355417 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:48.410537 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:48.410505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" event={"ID":"f5a3c2e5-f1c7-4ae6-8d22-4463ea7ec16c","Type":"ContainerStarted","Data":"a91d90fc99001b44fca6fe98d3d18fa77881a41f98cc5ef38745fd250b460e27"} Jul 07 10:32:48.449472 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:48.449380 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pgm7m" podStartSLOduration=1.017191646 podStartE2EDuration="3.449364202s" totalImagesPullingTime="2.432172556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:32:48.447860259 +0000 UTC m=+335.908395005" watchObservedRunningTime="2026-07-07 10:32:48.449364202 +0000 UTC m=+335.909898946" Jul 07 10:32:48.996819 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:48.996769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:49.131339 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:49.131299 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:51.199499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:51.199455 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:51.606560 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:51.606469 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:54.169244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:54.169200 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:54.245784 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:54.245750 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:55.574651 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.572675 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjlgp"] Jul 07 10:32:55.576414 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.576389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.579047 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.579017 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 10:32:55.580413 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.580395 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wvb4b\"" Jul 07 10:32:55.580759 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.580744 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 10:32:55.585281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.585260 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjlgp"] Jul 07 10:32:55.679729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.679694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22w8w\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-kube-api-access-22w8w\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.679926 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.679739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.781181 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.781151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22w8w\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-kube-api-access-22w8w\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.781337 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.781196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.791207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.791177 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.791336 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.791239 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22w8w\" (UniqueName: \"kubernetes.io/projected/a902219b-0139-4879-8ff2-1e01d8c9fc8f-kube-api-access-22w8w\") pod \"cert-manager-webhook-6486ffd997-rjlgp\" (UID: \"a902219b-0139-4879-8ff2-1e01d8c9fc8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:55.894039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:55.894011 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:56.021279 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:56.021244 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjlgp"] Jul 07 10:32:56.024399 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:32:56.024370 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda902219b_0139_4879_8ff2_1e01d8c9fc8f.slice/crio-a6ed0a1743ee96a0f0b54fd184b184044bd6bcc526d815c1c07f218575cb3993 WatchSource:0}: Error finding container a6ed0a1743ee96a0f0b54fd184b184044bd6bcc526d815c1c07f218575cb3993: Status 404 returned error can't find the container with id a6ed0a1743ee96a0f0b54fd184b184044bd6bcc526d815c1c07f218575cb3993 Jul 07 10:32:56.441706 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:56.441670 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" event={"ID":"a902219b-0139-4879-8ff2-1e01d8c9fc8f","Type":"ContainerStarted","Data":"a6ed0a1743ee96a0f0b54fd184b184044bd6bcc526d815c1c07f218575cb3993"} Jul 07 10:32:57.578873 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:57.578827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:57.677944 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:57.677905 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:59.250469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.250429 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:32:59.454804 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.454775 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" event={"ID":"a902219b-0139-4879-8ff2-1e01d8c9fc8f","Type":"ContainerStarted","Data":"1ea7b316e9e8b3b2207d9f750547e706678e8bd63a0fdf082ef3cb495d5c0afa"} Jul 07 10:32:59.454975 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.454910 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:32:59.481818 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.481753 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" podStartSLOduration=2.03273812 podStartE2EDuration="4.481735623s" totalImagesPullingTime="2.448997503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:32:59.478946627 +0000 UTC m=+346.939481374" watchObservedRunningTime="2026-07-07 10:32:59.481735623 +0000 UTC m=+346.942270368" Jul 07 10:32:59.690765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.690732 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4"] Jul 07 10:32:59.694069 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.694053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.698728 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.698710 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 10:32:59.700845 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.700825 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hhq46\"" Jul 07 10:32:59.700939 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.700851 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 10:32:59.710849 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.710787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdmrs\" (UniqueName: \"kubernetes.io/projected/43cafda0-7b90-45b1-971f-fc025ae8ac7e-kube-api-access-sdmrs\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.710957 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.710889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43cafda0-7b90-45b1-971f-fc025ae8ac7e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.712202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.712183 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4"] Jul 07 10:32:59.811476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.811443 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43cafda0-7b90-45b1-971f-fc025ae8ac7e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.811639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.811514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdmrs\" (UniqueName: \"kubernetes.io/projected/43cafda0-7b90-45b1-971f-fc025ae8ac7e-kube-api-access-sdmrs\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.811835 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.811792 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43cafda0-7b90-45b1-971f-fc025ae8ac7e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:32:59.824610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:32:59.824575 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdmrs\" (UniqueName: \"kubernetes.io/projected/43cafda0-7b90-45b1-971f-fc025ae8ac7e-kube-api-access-sdmrs\") pod \"openshift-lws-operator-fd8ccff4c-5dcl4\" (UID: \"43cafda0-7b90-45b1-971f-fc025ae8ac7e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:33:00.002387 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:00.002305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" Jul 07 10:33:00.148384 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:00.148348 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4"] Jul 07 10:33:00.149074 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:00.149050 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43cafda0_7b90_45b1_971f_fc025ae8ac7e.slice/crio-22d5deced5139a76c7bfe12feebe59a02e2cd1b707c190c169c0434bc9218e5f WatchSource:0}: Error finding container 22d5deced5139a76c7bfe12feebe59a02e2cd1b707c190c169c0434bc9218e5f: Status 404 returned error can't find the container with id 22d5deced5139a76c7bfe12feebe59a02e2cd1b707c190c169c0434bc9218e5f Jul 07 10:33:00.459948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:00.459907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" event={"ID":"43cafda0-7b90-45b1-971f-fc025ae8ac7e","Type":"ContainerStarted","Data":"22d5deced5139a76c7bfe12feebe59a02e2cd1b707c190c169c0434bc9218e5f"} Jul 07 10:33:03.476437 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:03.476401 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" event={"ID":"43cafda0-7b90-45b1-971f-fc025ae8ac7e","Type":"ContainerStarted","Data":"96c45418e133c206887b984ef99955f779b1d0fe29e3f27cb6520d5debb4100f"} Jul 07 10:33:03.499763 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:03.499711 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5dcl4" podStartSLOduration=1.932740758 podStartE2EDuration="4.499695444s" totalImagesPullingTime="2.566954686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:03.497228649 +0000 UTC m=+350.957763422" watchObservedRunningTime="2026-07-07 10:33:03.499695444 +0000 UTC m=+350.960230189" Jul 07 10:33:04.234762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:04.234729 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:04.336382 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:04.336343 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:05.462046 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:05.462017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rjlgp" Jul 07 10:33:06.123145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:06.123113 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:17.757091 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:17.757051 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:17.817167 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:17.817126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:21.185350 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:21.185313 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:21.277619 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:21.277583 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:21.775743 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:21.775695 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.112717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.112637 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.275907 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.275876 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq"] Jul 07 10:33:22.346207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.346173 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.349829 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.349786 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 10:33:22.350464 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.350449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 10:33:22.350737 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.350717 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 10:33:22.350862 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.350831 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9dx5k\"" Jul 07 10:33:22.351117 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.351099 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 10:33:22.363455 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.363401 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq"] Jul 07 10:33:22.400112 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.400083 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.424294 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.424263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-manifests\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.424435 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.424321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-apiservice-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.424435 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.424358 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-tmp\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.424435 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.424417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-webhook-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.424577 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.424452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvllb\" (UniqueName: \"kubernetes.io/projected/4631b4a6-fb49-43eb-bb12-ce5590455817-kube-api-access-wvllb\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.525409 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.525381 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-tmp\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.525566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.525429 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-webhook-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.525566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.525552 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvllb\" (UniqueName: \"kubernetes.io/projected/4631b4a6-fb49-43eb-bb12-ce5590455817-kube-api-access-wvllb\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.525650 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.525599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-manifests\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.525686 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.525655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-apiservice-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.526197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.526044 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-manifests\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.527769 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.527741 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4631b4a6-fb49-43eb-bb12-ce5590455817-tmp\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.527895 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.527874 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-webhook-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.528020 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.528006 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4631b4a6-fb49-43eb-bb12-ce5590455817-apiservice-cert\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.537472 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.537450 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvllb\" (UniqueName: \"kubernetes.io/projected/4631b4a6-fb49-43eb-bb12-ce5590455817-kube-api-access-wvllb\") pod \"opendatahub-operator-controller-manager-586fcf4d5b-f94lq\" (UID: \"4631b4a6-fb49-43eb-bb12-ce5590455817\") " pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.593026 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.592987 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.656176 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.656142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:22.682701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.682656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.799777 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.799710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:22.853600 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:22.853573 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4631b4a6_fb49_43eb_bb12_ce5590455817.slice/crio-80386dad12f83da804fd7e20e0c104b003ba8abf91664ec75b090055e88d42a6 WatchSource:0}: Error finding container 80386dad12f83da804fd7e20e0c104b003ba8abf91664ec75b090055e88d42a6: Status 404 returned error can't find the container with id 80386dad12f83da804fd7e20e0c104b003ba8abf91664ec75b090055e88d42a6 Jul 07 10:33:22.858889 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.858868 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq"] Jul 07 10:33:22.929222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:22.929140 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:23.029894 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:23.029850 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:23.558610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:23.558574 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" event={"ID":"4631b4a6-fb49-43eb-bb12-ce5590455817","Type":"ContainerStarted","Data":"80386dad12f83da804fd7e20e0c104b003ba8abf91664ec75b090055e88d42a6"} Jul 07 10:33:26.457248 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:26.457213 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:26.574199 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:26.574155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" event={"ID":"4631b4a6-fb49-43eb-bb12-ce5590455817","Type":"ContainerStarted","Data":"074f72d50e43e9acc15e095bbed33ba05143ad5cf7a07dfb4fdef0c4c0fb7ab5"} Jul 07 10:33:26.577090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:26.577059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:27.579912 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:27.579870 2581 generic.go:356] "Generic (PLEG): container finished" podID="4631b4a6-fb49-43eb-bb12-ce5590455817" containerID="074f72d50e43e9acc15e095bbed33ba05143ad5cf7a07dfb4fdef0c4c0fb7ab5" exitCode=0 Jul 07 10:33:27.580340 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:27.579953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" event={"ID":"4631b4a6-fb49-43eb-bb12-ce5590455817","Type":"ContainerDied","Data":"074f72d50e43e9acc15e095bbed33ba05143ad5cf7a07dfb4fdef0c4c0fb7ab5"} Jul 07 10:33:28.585596 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:28.585564 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" event={"ID":"4631b4a6-fb49-43eb-bb12-ce5590455817","Type":"ContainerStarted","Data":"0ea085a3ef166697d653794d492720da2eb0698a21bbc0a3761010e95f76aa3b"} Jul 07 10:33:28.586039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:28.585707 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:28.613541 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:28.613493 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" podStartSLOduration=2.181069197 podStartE2EDuration="6.613476971s" totalImagesPullingTime="3.36872743s" totalInitContainerRuntime="1.063680344s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:28.612044985 +0000 UTC m=+376.072579745" watchObservedRunningTime="2026-07-07 10:33:28.613476971 +0000 UTC m=+376.074011716" Jul 07 10:33:39.592154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:39.592120 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-586fcf4d5b-f94lq" Jul 07 10:33:41.411891 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:41.411854 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:41.630738 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:41.630702 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:42.127487 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.127452 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:42.136340 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.136312 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c48467d48-ml746"] Jul 07 10:33:42.139463 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.139446 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.144076 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.144057 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 10:33:42.144171 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.144060 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 10:33:42.162951 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.162924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c48467d48-ml746"] Jul 07 10:33:42.188765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.188728 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tls-certs\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.188765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.188764 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tmp\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.189002 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.188792 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2xfx\" (UniqueName: \"kubernetes.io/projected/d8825205-6632-4ee8-b4e7-93a2f856f0eb-kube-api-access-x2xfx\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.289923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.289892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tls-certs\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.290105 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.289931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tmp\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.290105 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.290051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2xfx\" (UniqueName: \"kubernetes.io/projected/d8825205-6632-4ee8-b4e7-93a2f856f0eb-kube-api-access-x2xfx\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.292327 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.292299 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tmp\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.292560 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.292544 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8825205-6632-4ee8-b4e7-93a2f856f0eb-tls-certs\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.334306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.334275 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:42.338827 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.338779 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2xfx\" (UniqueName: \"kubernetes.io/projected/d8825205-6632-4ee8-b4e7-93a2f856f0eb-kube-api-access-x2xfx\") pod \"kube-auth-proxy-5c48467d48-ml746\" (UID: \"d8825205-6632-4ee8-b4e7-93a2f856f0eb\") " pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.448717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.448622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" Jul 07 10:33:42.475739 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.475709 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-74927"] Jul 07 10:33:42.478943 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.478920 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.491925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.491894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mj5v\" (UniqueName: \"kubernetes.io/projected/6c27825c-ada8-4a89-9734-b39cb4622193-kube-api-access-7mj5v\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.492048 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.491954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c27825c-ada8-4a89-9734-b39cb4622193-cert\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.514642 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.514597 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 10:33:42.514925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.514904 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-h2mhd\"" Jul 07 10:33:42.526984 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.526955 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-74927"] Jul 07 10:33:42.575263 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.575231 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:42.592672 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.592641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mj5v\" (UniqueName: \"kubernetes.io/projected/6c27825c-ada8-4a89-9734-b39cb4622193-kube-api-access-7mj5v\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.592818 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.592680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c27825c-ada8-4a89-9734-b39cb4622193-cert\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.595138 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.595115 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c27825c-ada8-4a89-9734-b39cb4622193-cert\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.629408 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.629387 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mj5v\" (UniqueName: \"kubernetes.io/projected/6c27825c-ada8-4a89-9734-b39cb4622193-kube-api-access-7mj5v\") pod \"odh-model-controller-7998c67dc4-74927\" (UID: \"6c27825c-ada8-4a89-9734-b39cb4622193\") " pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.634353 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.634278 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c48467d48-ml746"] Jul 07 10:33:42.636495 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:42.636471 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8825205_6632_4ee8_b4e7_93a2f856f0eb.slice/crio-b2a89717356c5def88d82fc0519fd2a9a6b73dbf5a04db48580d64a9b73469e2 WatchSource:0}: Error finding container b2a89717356c5def88d82fc0519fd2a9a6b73dbf5a04db48580d64a9b73469e2: Status 404 returned error can't find the container with id b2a89717356c5def88d82fc0519fd2a9a6b73dbf5a04db48580d64a9b73469e2 Jul 07 10:33:42.645972 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.645946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" event={"ID":"d8825205-6632-4ee8-b4e7-93a2f856f0eb","Type":"ContainerStarted","Data":"b2a89717356c5def88d82fc0519fd2a9a6b73dbf5a04db48580d64a9b73469e2"} Jul 07 10:33:42.783844 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.783735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:42.791743 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.791724 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:42.966789 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:42.966762 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-74927"] Jul 07 10:33:42.968471 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:42.968444 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c27825c_ada8_4a89_9734_b39cb4622193.slice/crio-fd0c8427ab7405b8f9dcc8dae2b5bfb52864c621ff22030bc879983f1bc1b82c WatchSource:0}: Error finding container fd0c8427ab7405b8f9dcc8dae2b5bfb52864c621ff22030bc879983f1bc1b82c: Status 404 returned error can't find the container with id fd0c8427ab7405b8f9dcc8dae2b5bfb52864c621ff22030bc879983f1bc1b82c Jul 07 10:33:43.043917 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:43.043834 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:43.652301 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:43.652263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-74927" event={"ID":"6c27825c-ada8-4a89-9734-b39cb4622193","Type":"ContainerStarted","Data":"fd0c8427ab7405b8f9dcc8dae2b5bfb52864c621ff22030bc879983f1bc1b82c"} Jul 07 10:33:43.830219 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:43.830178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:44.137494 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.137461 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:44.415469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.415384 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:44.419458 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.419437 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.423725 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.423684 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 10:33:44.424124 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.423949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 10:33:44.424124 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.423949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 10:33:44.424639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.424261 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 10:33:44.426566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.426543 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jcgdm\"" Jul 07 10:33:44.426946 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.426907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:44.450527 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.450497 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:44.509961 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.509926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.509973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8xmw\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.510023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.510063 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510135 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.510098 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510345 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.510150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.510345 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.510172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.524703 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.524626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:44.611503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611562 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611876 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611876 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.611876 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.611849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8xmw\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.612556 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.612486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.614206 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.614172 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.614317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.614252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.614688 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.614665 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.614948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.614930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.632968 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.632936 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.633088 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.633001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8xmw\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw\") pod \"istiod-openshift-gateway-7574459656-mkkdg\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:44.736492 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:44.736397 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:45.758744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:45.758705 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:46.754271 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:46.754241 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:46.757166 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:46.757138 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67836f39_5999_4772_b0db_856a6c43e9c3.slice/crio-3894393bb2cf34c8cdded13e798898e7dc664b486f57fe953d5678ac992ea123 WatchSource:0}: Error finding container 3894393bb2cf34c8cdded13e798898e7dc664b486f57fe953d5678ac992ea123: Status 404 returned error can't find the container with id 3894393bb2cf34c8cdded13e798898e7dc664b486f57fe953d5678ac992ea123 Jul 07 10:33:47.674079 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.674041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" event={"ID":"67836f39-5999-4772-b0db-856a6c43e9c3","Type":"ContainerStarted","Data":"3894393bb2cf34c8cdded13e798898e7dc664b486f57fe953d5678ac992ea123"} Jul 07 10:33:47.675698 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.675665 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-74927" event={"ID":"6c27825c-ada8-4a89-9734-b39cb4622193","Type":"ContainerStarted","Data":"3223f77a78002e979b90e8c90fb600f282bd0ff7281ec04a1b7d43ee93b0385f"} Jul 07 10:33:47.675842 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.675781 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:33:47.677419 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.677370 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" event={"ID":"d8825205-6632-4ee8-b4e7-93a2f856f0eb","Type":"ContainerStarted","Data":"fd850f6ba1d5a0f98bd89f4f44a1c10fd45cda7cfb6296c91d528e3a9008ee8e"} Jul 07 10:33:47.700443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.700380 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-74927" podStartSLOduration=2.057027711 podStartE2EDuration="5.700359539s" totalImagesPullingTime="3.643331828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:47.699754553 +0000 UTC m=+395.160289326" watchObservedRunningTime="2026-07-07 10:33:47.700359539 +0000 UTC m=+395.160894276" Jul 07 10:33:47.727114 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:47.727051 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c48467d48-ml746" podStartSLOduration=1.750358792 podStartE2EDuration="5.727030349s" totalImagesPullingTime="3.976671557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:47.724877959 +0000 UTC m=+395.185412705" watchObservedRunningTime="2026-07-07 10:33:47.727030349 +0000 UTC m=+395.187565130" Jul 07 10:33:49.063977 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.063927 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:33:49.064310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.063999 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:33:49.689562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.689527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" event={"ID":"67836f39-5999-4772-b0db-856a6c43e9c3","Type":"ContainerStarted","Data":"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4"} Jul 07 10:33:49.689748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.689643 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:49.759483 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.759424 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" podStartSLOduration=3.455051811 podStartE2EDuration="5.759398695s" totalImagesPullingTime="2.304346884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:49.757540848 +0000 UTC m=+397.218075623" watchObservedRunningTime="2026-07-07 10:33:49.759398695 +0000 UTC m=+397.219933440" Jul 07 10:33:49.914926 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:49.914895 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:50.099174 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:50.099094 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:50.697635 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:50.697599 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-mkkdg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 10:33:50.697789 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:50.697654 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 10:33:51.587410 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.587375 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h"] Jul 07 10:33:51.590709 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.590683 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.597297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.597276 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jxb5x\"" Jul 07 10:33:51.679589 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679555 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.679773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679611 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.679773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v54nw\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-kube-api-access-v54nw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.679773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679748 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.679927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.679927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.680002 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.680002 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.679978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.680070 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.680008 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.727782 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.727752 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:51.780941 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.780909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.780951 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.780980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v54nw\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-kube-api-access-v54nw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781382 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781446 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781429 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781507 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781484 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781614 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781713 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781684 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.781853 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.781747 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.783370 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.783350 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.783768 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.783748 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.837158 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.837129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.838657 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.838592 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h"] Jul 07 10:33:51.867511 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.867486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v54nw\" (UniqueName: \"kubernetes.io/projected/95b07372-b3d8-4bf5-b464-ca2746b7dc85-kube-api-access-v54nw\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4f4t7h\" (UID: \"95b07372-b3d8-4bf5-b464-ca2746b7dc85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:51.900595 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:51.900561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:52.020196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:52.020164 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:52.101946 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:52.101920 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h"] Jul 07 10:33:52.104561 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:52.104535 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b07372_b3d8_4bf5_b464_ca2746b7dc85.slice/crio-a65aa6803d416a60eb2dbd971dd29982b20d0d6343faebdb15ac4591aecdd715 WatchSource:0}: Error finding container a65aa6803d416a60eb2dbd971dd29982b20d0d6343faebdb15ac4591aecdd715: Status 404 returned error can't find the container with id a65aa6803d416a60eb2dbd971dd29982b20d0d6343faebdb15ac4591aecdd715 Jul 07 10:33:52.237189 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:52.237157 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:52.704863 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:52.704825 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" event={"ID":"95b07372-b3d8-4bf5-b464-ca2746b7dc85","Type":"ContainerStarted","Data":"a65aa6803d416a60eb2dbd971dd29982b20d0d6343faebdb15ac4591aecdd715"} Jul 07 10:33:53.696895 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:53.696864 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:54.551499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.551465 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:33:54.551721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.551536 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:33:54.551721 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.551565 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:33:54.716221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.716183 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" event={"ID":"95b07372-b3d8-4bf5-b464-ca2746b7dc85","Type":"ContainerStarted","Data":"fd8fd2388575b89f94d56baa0ca0581748e67fa6238b2cfa0b0ecefd38b3ec66"} Jul 07 10:33:54.873892 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.873833 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" podStartSLOduration=1.429300508 podStartE2EDuration="3.873817233s" totalImagesPullingTime="2.444516725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:33:54.873319175 +0000 UTC m=+402.333853935" watchObservedRunningTime="2026-07-07 10:33:54.873817233 +0000 UTC m=+402.334351977" Jul 07 10:33:54.901634 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:54.901608 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:55.717762 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.717724 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:55.718244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.718009 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" containerName="discovery" containerID="cri-o://31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4" gracePeriod=30 Jul 07 10:33:55.724212 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.724183 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:55.906879 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.906849 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:55.954813 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.954763 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-swm8t"] Jul 07 10:33:55.960653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.960631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:55.970864 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.970765 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qjtll\"" Jul 07 10:33:55.974226 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.974197 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:55.974424 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.974407 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 10:33:55.974922 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.974901 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:55.982835 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:55.982784 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-swm8t"] Jul 07 10:33:56.019250 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019224 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019426 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019257 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8xmw\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019426 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019279 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019426 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019311 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019426 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019401 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019437 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019501 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig\") pod \"67836f39-5999-4772-b0db-856a6c43e9c3\" (UID: \"67836f39-5999-4772-b0db-856a6c43e9c3\") " Jul 07 10:33:56.019747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019723 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.019747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019733 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:33:56.019891 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvlq\" (UniqueName: \"kubernetes.io/projected/ff3a74ee-75bb-4ad1-ac14-733562b9197d-kube-api-access-njvlq\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.019962 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.019948 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-ca-configmap\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.022087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022059 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:33:56.022213 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022092 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs" (OuterVolumeSpecName: "local-certs") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:33:56.022280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022252 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw" (OuterVolumeSpecName: "kube-api-access-m8xmw") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "kube-api-access-m8xmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:33:56.022337 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022284 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts" (OuterVolumeSpecName: "cacerts") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:33:56.022402 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022370 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:33:56.022510 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.022496 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "67836f39-5999-4772-b0db-856a6c43e9c3" (UID: "67836f39-5999-4772-b0db-856a6c43e9c3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:33:56.048085 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.048057 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz"] Jul 07 10:33:56.048400 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.048375 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" containerName="discovery" Jul 07 10:33:56.048400 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.048393 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" containerName="discovery" Jul 07 10:33:56.048588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.048449 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" containerName="discovery" Jul 07 10:33:56.053736 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.053716 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.059865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.059792 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 10:33:56.077401 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.077380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wz7ln\"" Jul 07 10:33:56.103979 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.103951 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz"] Jul 07 10:33:56.121179 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/498a8dba-7f57-449d-a0c3-051d6b00e683-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.121179 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121181 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.121344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvlq\" (UniqueName: \"kubernetes.io/projected/ff3a74ee-75bb-4ad1-ac14-733562b9197d-kube-api-access-njvlq\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.121344 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:33:56.121310 2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 07 10:33:56.121427 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:33:56.121377 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert podName:ff3a74ee-75bb-4ad1-ac14-733562b9197d nodeName:}" failed. No retries permitted until 2026-07-07 10:33:56.621353854 +0000 UTC m=+404.081888594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert") pod "kserve-controller-manager-d6f4b65b8-swm8t" (UID: "ff3a74ee-75bb-4ad1-ac14-733562b9197d") : secret "kserve-webhook-server-cert" not found Jul 07 10:33:56.121427 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121401 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7dj\" (UniqueName: \"kubernetes.io/projected/498a8dba-7f57-449d-a0c3-051d6b00e683-kube-api-access-5z7dj\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.121563 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121546 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-kubeconfig\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.121610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121566 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-istio-csr-dns-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.121610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121577 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8xmw\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-kube-api-access-m8xmw\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.121610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121587 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/67836f39-5999-4772-b0db-856a6c43e9c3-local-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.121610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121595 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67836f39-5999-4772-b0db-856a6c43e9c3-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.121610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.121602 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/67836f39-5999-4772-b0db-856a6c43e9c3-cacerts\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:33:56.179601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.179574 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvlq\" (UniqueName: \"kubernetes.io/projected/ff3a74ee-75bb-4ad1-ac14-733562b9197d-kube-api-access-njvlq\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.213092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.213060 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:56.222285 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.222206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/498a8dba-7f57-449d-a0c3-051d6b00e683-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.222414 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.222298 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7dj\" (UniqueName: \"kubernetes.io/projected/498a8dba-7f57-449d-a0c3-051d6b00e683-kube-api-access-5z7dj\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.225268 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.225245 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/498a8dba-7f57-449d-a0c3-051d6b00e683-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.263932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.263904 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7dj\" (UniqueName: \"kubernetes.io/projected/498a8dba-7f57-449d-a0c3-051d6b00e683-kube-api-access-5z7dj\") pod \"llmisvc-controller-manager-777c6bd7d5-lvrtz\" (UID: \"498a8dba-7f57-449d-a0c3-051d6b00e683\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.363330 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.363287 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:33:56.397228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.397192 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:56.569336 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:33:56.569295 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod498a8dba_7f57_449d_a0c3_051d6b00e683.slice/crio-708ff60d751a5d144927a8e2e4f3ce3490f2fe13c55ee7a1be5d862918df237d WatchSource:0}: Error finding container 708ff60d751a5d144927a8e2e4f3ce3490f2fe13c55ee7a1be5d862918df237d: Status 404 returned error can't find the container with id 708ff60d751a5d144927a8e2e4f3ce3490f2fe13c55ee7a1be5d862918df237d Jul 07 10:33:56.569514 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.569390 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz"] Jul 07 10:33:56.631567 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.631533 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.634226 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.634197 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff3a74ee-75bb-4ad1-ac14-733562b9197d-cert\") pod \"kserve-controller-manager-d6f4b65b8-swm8t\" (UID: \"ff3a74ee-75bb-4ad1-ac14-733562b9197d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.727134 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.727095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" event={"ID":"498a8dba-7f57-449d-a0c3-051d6b00e683","Type":"ContainerStarted","Data":"708ff60d751a5d144927a8e2e4f3ce3490f2fe13c55ee7a1be5d862918df237d"} Jul 07 10:33:56.728445 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.728417 2581 generic.go:356] "Generic (PLEG): container finished" podID="67836f39-5999-4772-b0db-856a6c43e9c3" containerID="31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4" exitCode=0 Jul 07 10:33:56.728554 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.728470 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" Jul 07 10:33:56.728554 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.728502 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" event={"ID":"67836f39-5999-4772-b0db-856a6c43e9c3","Type":"ContainerDied","Data":"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4"} Jul 07 10:33:56.728675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.728554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg" event={"ID":"67836f39-5999-4772-b0db-856a6c43e9c3","Type":"ContainerDied","Data":"3894393bb2cf34c8cdded13e798898e7dc664b486f57fe953d5678ac992ea123"} Jul 07 10:33:56.728675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.728581 2581 scope.go:117] "RemoveContainer" containerID="31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4" Jul 07 10:33:56.729187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.729171 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:56.730142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.730119 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4f4t7h" Jul 07 10:33:56.737775 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.737759 2581 scope.go:117] "RemoveContainer" containerID="31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4" Jul 07 10:33:56.738043 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:33:56.738026 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4\": container with ID starting with 31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4 not found: ID does not exist" containerID="31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4" Jul 07 10:33:56.738108 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.738050 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4"} err="failed to get container status \"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4\": rpc error: code = NotFound desc = could not find container \"31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4\": container with ID starting with 31cb0d9f6c2ad833b4399e4b8cd2ef07b26c9b86a8a09752f993cc1b87a7e1d4 not found: ID does not exist" Jul 07 10:33:56.866894 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.866752 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:56.879344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.879317 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:33:56.884825 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:56.884780 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mkkdg"] Jul 07 10:33:57.078579 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:57.078550 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67836f39-5999-4772-b0db-856a6c43e9c3" path="/var/lib/kubelet/pods/67836f39-5999-4772-b0db-856a6c43e9c3/volumes" Jul 07 10:33:57.098724 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:57.098670 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-swm8t"] Jul 07 10:33:57.276196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:57.276166 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:57.396149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:57.396111 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:33:57.740373 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:57.740126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" event={"ID":"ff3a74ee-75bb-4ad1-ac14-733562b9197d","Type":"ContainerStarted","Data":"7df18ffa4e0ff069097de98f409173f65f140db932934a95a1b74fa04765497e"} Jul 07 10:33:58.686674 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:33:58.686649 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-74927" Jul 07 10:34:00.758164 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.758109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" event={"ID":"ff3a74ee-75bb-4ad1-ac14-733562b9197d","Type":"ContainerStarted","Data":"1e064de897c238e5035566bdeeb347f1cd5e24ef88fe24ec0a0943ed9417806b"} Jul 07 10:34:00.758612 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.758196 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:34:00.759719 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.759695 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" event={"ID":"498a8dba-7f57-449d-a0c3-051d6b00e683","Type":"ContainerStarted","Data":"87451110fdb0d57d6c7b0ba0f38eca6197fbf1b908bb9284ea0fa3cc44eec7cf"} Jul 07 10:34:00.759856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.759751 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:34:00.874865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.874790 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" podStartSLOduration=3.018235554 podStartE2EDuration="5.874771706s" totalImagesPullingTime="2.856536152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:34:00.815638315 +0000 UTC m=+408.276173060" watchObservedRunningTime="2026-07-07 10:34:00.874771706 +0000 UTC m=+408.335306451" Jul 07 10:34:00.875047 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:00.874904 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" podStartSLOduration=2.487222575 podStartE2EDuration="5.874897676s" totalImagesPullingTime="3.387675101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:34:00.874408105 +0000 UTC m=+408.334942851" watchObservedRunningTime="2026-07-07 10:34:00.874897676 +0000 UTC m=+408.335432421" Jul 07 10:34:15.643904 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:15.643868 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:15.715969 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:15.715934 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:31.769332 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:31.769300 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lvrtz" Jul 07 10:34:31.772216 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:31.772196 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-swm8t" Jul 07 10:34:43.041765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.041672 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:43.202013 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.201965 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:43.202226 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.202079 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:43.205820 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.205743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:43.217144 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.217118 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 10:34:43.218718 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.218554 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-s2m7d\"" Jul 07 10:34:43.218861 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.218788 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 10:34:43.220384 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.220349 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:43.237965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.237942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzth4\" (UniqueName: \"kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4\") pod \"kuadrant-operator-catalog-z57kg\" (UID: \"7608958f-4163-4859-a4d1-dbd89408b1bd\") " pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:43.339229 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.339146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzth4\" (UniqueName: \"kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4\") pod \"kuadrant-operator-catalog-z57kg\" (UID: \"7608958f-4163-4859-a4d1-dbd89408b1bd\") " pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:43.343497 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.343470 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:43.348603 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.348587 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzth4\" (UniqueName: \"kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4\") pod \"kuadrant-operator-catalog-z57kg\" (UID: \"7608958f-4163-4859-a4d1-dbd89408b1bd\") " pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:43.488360 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.488322 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:43.488600 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.488588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:43.627614 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.627589 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:43.630067 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:34:43.630041 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7608958f_4163_4859_a4d1_dbd89408b1bd.slice/crio-d022816cd04d9a828788330a1a3c6cd6fdbd1bd5a542db9ea94df79455ce9b49 WatchSource:0}: Error finding container d022816cd04d9a828788330a1a3c6cd6fdbd1bd5a542db9ea94df79455ce9b49: Status 404 returned error can't find the container with id d022816cd04d9a828788330a1a3c6cd6fdbd1bd5a542db9ea94df79455ce9b49 Jul 07 10:34:43.717037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.717003 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x4dg6"] Jul 07 10:34:43.720868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.720851 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:43.738943 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.738915 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x4dg6"] Jul 07 10:34:43.743025 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.743001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg226\" (UniqueName: \"kubernetes.io/projected/e3d71b29-d61e-48ce-a94a-0f18c5e95ea1-kube-api-access-rg226\") pod \"kuadrant-operator-catalog-x4dg6\" (UID: \"e3d71b29-d61e-48ce-a94a-0f18c5e95ea1\") " pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:43.843607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.843577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg226\" (UniqueName: \"kubernetes.io/projected/e3d71b29-d61e-48ce-a94a-0f18c5e95ea1-kube-api-access-rg226\") pod \"kuadrant-operator-catalog-x4dg6\" (UID: \"e3d71b29-d61e-48ce-a94a-0f18c5e95ea1\") " pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:43.852525 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.852493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg226\" (UniqueName: \"kubernetes.io/projected/e3d71b29-d61e-48ce-a94a-0f18c5e95ea1-kube-api-access-rg226\") pod \"kuadrant-operator-catalog-x4dg6\" (UID: \"e3d71b29-d61e-48ce-a94a-0f18c5e95ea1\") " pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:43.938867 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.938760 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:43.951415 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:43.951387 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" event={"ID":"7608958f-4163-4859-a4d1-dbd89408b1bd","Type":"ContainerStarted","Data":"d022816cd04d9a828788330a1a3c6cd6fdbd1bd5a542db9ea94df79455ce9b49"} Jul 07 10:34:44.030269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:44.030236 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:44.179950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:44.179763 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:34:44.179950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:44.179928 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x4dg6"] Jul 07 10:34:44.182018 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:34:44.181995 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d71b29_d61e_48ce_a94a_0f18c5e95ea1.slice/crio-cfc140ee3828f4ae5e06a4b74a2876a2b30097bb384ef60d28587210e977e8d3 WatchSource:0}: Error finding container cfc140ee3828f4ae5e06a4b74a2876a2b30097bb384ef60d28587210e977e8d3: Status 404 returned error can't find the container with id cfc140ee3828f4ae5e06a4b74a2876a2b30097bb384ef60d28587210e977e8d3 Jul 07 10:34:44.956942 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:44.956910 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" event={"ID":"e3d71b29-d61e-48ce-a94a-0f18c5e95ea1","Type":"ContainerStarted","Data":"cfc140ee3828f4ae5e06a4b74a2876a2b30097bb384ef60d28587210e977e8d3"} Jul 07 10:34:45.965222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:45.965179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" event={"ID":"e3d71b29-d61e-48ce-a94a-0f18c5e95ea1","Type":"ContainerStarted","Data":"ab5686c4d6793fafe8d5546adcf4448ac281fe0f8a63d7b39641f1ddd11cb0af"} Jul 07 10:34:45.966885 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:45.966854 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" event={"ID":"7608958f-4163-4859-a4d1-dbd89408b1bd","Type":"ContainerStarted","Data":"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7"} Jul 07 10:34:45.967006 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:45.966982 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" podUID="7608958f-4163-4859-a4d1-dbd89408b1bd" containerName="registry-server" containerID="cri-o://5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7" gracePeriod=2 Jul 07 10:34:45.995500 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:45.995424 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" podStartSLOduration=1.3413855639999999 podStartE2EDuration="2.995405437s" totalImagesPullingTime="1.654019873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:34:45.992575892 +0000 UTC m=+453.453110655" watchObservedRunningTime="2026-07-07 10:34:45.995405437 +0000 UTC m=+453.455940181" Jul 07 10:34:46.035996 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.035830 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" podStartSLOduration=0.832447367 podStartE2EDuration="3.035787847s" totalImagesPullingTime="2.20334048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:34:46.032731098 +0000 UTC m=+453.493265863" watchObservedRunningTime="2026-07-07 10:34:46.035787847 +0000 UTC m=+453.496322593" Jul 07 10:34:46.212755 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.212727 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:46.269837 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.269787 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzth4\" (UniqueName: \"kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4\") pod \"7608958f-4163-4859-a4d1-dbd89408b1bd\" (UID: \"7608958f-4163-4859-a4d1-dbd89408b1bd\") " Jul 07 10:34:46.272078 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.272050 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4" (OuterVolumeSpecName: "kube-api-access-hzth4") pod "7608958f-4163-4859-a4d1-dbd89408b1bd" (UID: "7608958f-4163-4859-a4d1-dbd89408b1bd"). InnerVolumeSpecName "kube-api-access-hzth4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:34:46.370422 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.370396 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzth4\" (UniqueName: \"kubernetes.io/projected/7608958f-4163-4859-a4d1-dbd89408b1bd-kube-api-access-hzth4\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:34:46.972489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.972455 2581 generic.go:356] "Generic (PLEG): container finished" podID="7608958f-4163-4859-a4d1-dbd89408b1bd" containerID="5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7" exitCode=0 Jul 07 10:34:46.972986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.972506 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" Jul 07 10:34:46.972986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.972543 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" event={"ID":"7608958f-4163-4859-a4d1-dbd89408b1bd","Type":"ContainerDied","Data":"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7"} Jul 07 10:34:46.972986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.972595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z57kg" event={"ID":"7608958f-4163-4859-a4d1-dbd89408b1bd","Type":"ContainerDied","Data":"d022816cd04d9a828788330a1a3c6cd6fdbd1bd5a542db9ea94df79455ce9b49"} Jul 07 10:34:46.972986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.972621 2581 scope.go:117] "RemoveContainer" containerID="5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7" Jul 07 10:34:46.981746 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.981732 2581 scope.go:117] "RemoveContainer" containerID="5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7" Jul 07 10:34:46.982019 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:34:46.982001 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7\": container with ID starting with 5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7 not found: ID does not exist" containerID="5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7" Jul 07 10:34:46.982090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.982026 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7"} err="failed to get container status \"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7\": rpc error: code = NotFound desc = could not find container \"5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7\": container with ID starting with 5908c318df4faf7688f2f7754c23793303427992278df4c010d0192fcd8d62f7 not found: ID does not exist" Jul 07 10:34:46.996231 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:46.996208 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:47.000281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:47.000259 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z57kg"] Jul 07 10:34:47.082919 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:47.082885 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7608958f-4163-4859-a4d1-dbd89408b1bd" path="/var/lib/kubelet/pods/7608958f-4163-4859-a4d1-dbd89408b1bd/volumes" Jul 07 10:34:54.030503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:54.030463 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:54.030954 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:54.030543 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:54.053870 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:54.053840 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:34:55.032887 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:34:55.032856 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-x4dg6" Jul 07 10:35:13.276550 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:13.276448 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:13.898388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:13.898346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:17.334928 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.334884 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tqlts"] Jul 07 10:35:17.335456 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.335380 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7608958f-4163-4859-a4d1-dbd89408b1bd" containerName="registry-server" Jul 07 10:35:17.335456 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.335397 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7608958f-4163-4859-a4d1-dbd89408b1bd" containerName="registry-server" Jul 07 10:35:17.335548 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.335470 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7608958f-4163-4859-a4d1-dbd89408b1bd" containerName="registry-server" Jul 07 10:35:17.339781 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.339762 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:17.343296 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.343279 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5rqmz\"" Jul 07 10:35:17.360403 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.357514 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tqlts"] Jul 07 10:35:17.443130 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.443097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vm5z\" (UniqueName: \"kubernetes.io/projected/4c890e0f-6fd9-46ce-8c01-4f52796c6637-kube-api-access-7vm5z\") pod \"authorino-operator-75cc8d5cb7-tqlts\" (UID: \"4c890e0f-6fd9-46ce-8c01-4f52796c6637\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:17.544502 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.544467 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vm5z\" (UniqueName: \"kubernetes.io/projected/4c890e0f-6fd9-46ce-8c01-4f52796c6637-kube-api-access-7vm5z\") pod \"authorino-operator-75cc8d5cb7-tqlts\" (UID: \"4c890e0f-6fd9-46ce-8c01-4f52796c6637\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:17.556386 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.556355 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vm5z\" (UniqueName: \"kubernetes.io/projected/4c890e0f-6fd9-46ce-8c01-4f52796c6637-kube-api-access-7vm5z\") pod \"authorino-operator-75cc8d5cb7-tqlts\" (UID: \"4c890e0f-6fd9-46ce-8c01-4f52796c6637\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:17.661529 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.661498 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:17.824051 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:17.823968 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tqlts"] Jul 07 10:35:17.826229 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:35:17.826201 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c890e0f_6fd9_46ce_8c01_4f52796c6637.slice/crio-1323817e5b63ab0d992e449ef0ff64ffc44a3c9761ba07494d5474567333712d WatchSource:0}: Error finding container 1323817e5b63ab0d992e449ef0ff64ffc44a3c9761ba07494d5474567333712d: Status 404 returned error can't find the container with id 1323817e5b63ab0d992e449ef0ff64ffc44a3c9761ba07494d5474567333712d Jul 07 10:35:18.108228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:18.108141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" event={"ID":"4c890e0f-6fd9-46ce-8c01-4f52796c6637","Type":"ContainerStarted","Data":"1323817e5b63ab0d992e449ef0ff64ffc44a3c9761ba07494d5474567333712d"} Jul 07 10:35:18.596783 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:18.596739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:20.633734 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.633695 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh"] Jul 07 10:35:20.635938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.635913 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:20.637326 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.637309 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:20.644074 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.644056 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j64cs\"" Jul 07 10:35:20.661623 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.661597 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh"] Jul 07 10:35:20.776965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.776931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7hd6\" (UniqueName: \"kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6\") pod \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" (UID: \"e4818d64-7ee5-487b-8998-162de3ccdc6f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:20.778255 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.778221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:20.878169 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.878133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7hd6\" (UniqueName: \"kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6\") pod \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" (UID: \"e4818d64-7ee5-487b-8998-162de3ccdc6f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:20.898959 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.898888 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7hd6\" (UniqueName: \"kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6\") pod \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" (UID: \"e4818d64-7ee5-487b-8998-162de3ccdc6f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:20.946890 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:20.946856 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:21.084469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.084442 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh"] Jul 07 10:35:21.087213 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:35:21.087186 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4818d64_7ee5_487b_8998_162de3ccdc6f.slice/crio-494944f0ac5b0dc0b6102d05958051c66114d13d0c7a5d734c557b2b54073548 WatchSource:0}: Error finding container 494944f0ac5b0dc0b6102d05958051c66114d13d0c7a5d734c557b2b54073548: Status 404 returned error can't find the container with id 494944f0ac5b0dc0b6102d05958051c66114d13d0c7a5d734c557b2b54073548 Jul 07 10:35:21.123612 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.123580 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" event={"ID":"4c890e0f-6fd9-46ce-8c01-4f52796c6637","Type":"ContainerStarted","Data":"f2fbf4e3edcca3735788c4ccc599a6cc6be50abc16f221890789ce6b75c3ab5d"} Jul 07 10:35:21.123821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.123705 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:21.124932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.124912 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" event={"ID":"e4818d64-7ee5-487b-8998-162de3ccdc6f","Type":"ContainerStarted","Data":"494944f0ac5b0dc0b6102d05958051c66114d13d0c7a5d734c557b2b54073548"} Jul 07 10:35:21.146834 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.146751 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" podStartSLOduration=1.839222505 podStartE2EDuration="4.146736538s" totalImagesPullingTime="2.307514033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:35:21.145608961 +0000 UTC m=+488.606143748" watchObservedRunningTime="2026-07-07 10:35:21.146736538 +0000 UTC m=+488.607271282" Jul 07 10:35:21.506523 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:21.506490 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:22.375149 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:22.375113 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:23.136260 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.136226 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" event={"ID":"e4818d64-7ee5-487b-8998-162de3ccdc6f","Type":"ContainerStarted","Data":"46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0"} Jul 07 10:35:23.136454 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.136340 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:23.159104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.159052 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" podStartSLOduration=1.2883642229999999 podStartE2EDuration="3.159034662s" totalImagesPullingTime="1.870670439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:35:23.157592287 +0000 UTC m=+490.618127045" watchObservedRunningTime="2026-07-07 10:35:23.159034662 +0000 UTC m=+490.619569407" Jul 07 10:35:23.788957 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.788921 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk"] Jul 07 10:35:23.792207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.792191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:23.800689 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.800661 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 10:35:23.801043 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.801022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-26kkr\"" Jul 07 10:35:23.804214 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.804182 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxvch\" (UniqueName: \"kubernetes.io/projected/952907dc-4a8b-4996-bbcf-5bd1ee3590a6-kube-api-access-jxvch\") pod \"dns-operator-controller-manager-864dd78c4c-dtstk\" (UID: \"952907dc-4a8b-4996-bbcf-5bd1ee3590a6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:23.820093 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.820056 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk"] Jul 07 10:35:23.905172 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.905135 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxvch\" (UniqueName: \"kubernetes.io/projected/952907dc-4a8b-4996-bbcf-5bd1ee3590a6-kube-api-access-jxvch\") pod \"dns-operator-controller-manager-864dd78c4c-dtstk\" (UID: \"952907dc-4a8b-4996-bbcf-5bd1ee3590a6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:23.916366 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:23.916338 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxvch\" (UniqueName: \"kubernetes.io/projected/952907dc-4a8b-4996-bbcf-5bd1ee3590a6-kube-api-access-jxvch\") pod \"dns-operator-controller-manager-864dd78c4c-dtstk\" (UID: \"952907dc-4a8b-4996-bbcf-5bd1ee3590a6\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:24.103926 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:24.103844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:24.257923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:24.257893 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk"] Jul 07 10:35:24.259750 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:35:24.259724 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952907dc_4a8b_4996_bbcf_5bd1ee3590a6.slice/crio-035543ad1de06b822f6a81119e0486a79fc4fbf7f5a78382f9b9e6fd50b541e7 WatchSource:0}: Error finding container 035543ad1de06b822f6a81119e0486a79fc4fbf7f5a78382f9b9e6fd50b541e7: Status 404 returned error can't find the container with id 035543ad1de06b822f6a81119e0486a79fc4fbf7f5a78382f9b9e6fd50b541e7 Jul 07 10:35:24.549418 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:24.549383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:25.032039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:25.032006 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:25.146720 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:25.146687 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" event={"ID":"952907dc-4a8b-4996-bbcf-5bd1ee3590a6","Type":"ContainerStarted","Data":"035543ad1de06b822f6a81119e0486a79fc4fbf7f5a78382f9b9e6fd50b541e7"} Jul 07 10:35:27.157345 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:27.157310 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" event={"ID":"952907dc-4a8b-4996-bbcf-5bd1ee3590a6","Type":"ContainerStarted","Data":"b23b43c4f803cab78312673e1dd4a727cb46f5cc8e4543d1475d06865d5b3da0"} Jul 07 10:35:27.157781 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:27.157376 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:27.178417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:27.178363 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" podStartSLOduration=2.289067492 podStartE2EDuration="4.178346306s" totalImagesPullingTime="1.889278814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:35:27.177701981 +0000 UTC m=+494.638236742" watchObservedRunningTime="2026-07-07 10:35:27.178346306 +0000 UTC m=+494.638881051" Jul 07 10:35:28.191840 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:28.191774 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:29.097901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.097868 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h"] Jul 07 10:35:29.101563 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.101538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.105235 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.105214 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mx77n\"" Jul 07 10:35:29.118861 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.118834 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h"] Jul 07 10:35:29.155959 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.155923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pknrv\" (UniqueName: \"kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.156116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.156041 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.257066 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.257034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.257066 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.257075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pknrv\" (UniqueName: \"kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.257496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.257419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.274472 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.274446 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pknrv\" (UniqueName: \"kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.411388 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.411353 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:29.577042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:29.577013 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h"] Jul 07 10:35:29.577200 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:35:29.577179 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb8977a_127d_4f82_adc1_7bae2b74c62c.slice/crio-3254ab6ce9acbf466b86f335ae19589207a007edd234e70f5348755f393d8774 WatchSource:0}: Error finding container 3254ab6ce9acbf466b86f335ae19589207a007edd234e70f5348755f393d8774: Status 404 returned error can't find the container with id 3254ab6ce9acbf466b86f335ae19589207a007edd234e70f5348755f393d8774 Jul 07 10:35:30.110413 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:30.110381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:30.173551 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:30.173514 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" event={"ID":"eeb8977a-127d-4f82-adc1-7bae2b74c62c","Type":"ContainerStarted","Data":"3254ab6ce9acbf466b86f335ae19589207a007edd234e70f5348755f393d8774"} Jul 07 10:35:30.209326 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:30.209293 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:30.489495 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:30.489450 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:30.591979 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:30.591938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:32.132831 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:32.132780 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tqlts" Jul 07 10:35:33.719143 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:33.719069 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:33.851416 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:33.851328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:34.143068 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:34.143038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:35.202313 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:35.202270 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" event={"ID":"eeb8977a-127d-4f82-adc1-7bae2b74c62c","Type":"ContainerStarted","Data":"5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c"} Jul 07 10:35:35.202717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:35.202408 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:35.234452 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:35.230072 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" podStartSLOduration=0.812894075 podStartE2EDuration="6.23004764s" totalImagesPullingTime="5.417153565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:35:35.227007043 +0000 UTC m=+502.687541808" watchObservedRunningTime="2026-07-07 10:35:35.23004764 +0000 UTC m=+502.690582388" Jul 07 10:35:35.745148 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:35.745108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:36.113358 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:36.113272 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:37.767234 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.767203 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd"] Jul 07 10:35:37.770634 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.770607 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.773453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.773416 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 10:35:37.773453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.773440 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 10:35:37.773663 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.773501 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cp7dp\"" Jul 07 10:35:37.782925 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.782902 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd"] Jul 07 10:35:37.790536 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.790511 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:37.835321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.835289 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/abf3b900-88a7-4506-84e7-d708564acbca-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.835485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.835335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9hlh\" (UniqueName: \"kubernetes.io/projected/abf3b900-88a7-4506-84e7-d708564acbca-kube-api-access-m9hlh\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.835485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.835356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.936212 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.936171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/abf3b900-88a7-4506-84e7-d708564acbca-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.936382 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.936229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9hlh\" (UniqueName: \"kubernetes.io/projected/abf3b900-88a7-4506-84e7-d708564acbca-kube-api-access-m9hlh\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.936382 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.936253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.936382 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:37.936379 2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 07 10:35:37.936498 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:37.936440 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert podName:abf3b900-88a7-4506-84e7-d708564acbca nodeName:}" failed. No retries permitted until 2026-07-07 10:35:38.4364257 +0000 UTC m=+505.896960422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-kv8xd" (UID: "abf3b900-88a7-4506-84e7-d708564acbca") : secret "plugin-serving-cert" not found Jul 07 10:35:37.936872 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.936853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/abf3b900-88a7-4506-84e7-d708564acbca-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:37.947501 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:37.947473 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9hlh\" (UniqueName: \"kubernetes.io/projected/abf3b900-88a7-4506-84e7-d708564acbca-kube-api-access-m9hlh\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:38.111674 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.111578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:38.165221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.165193 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dtstk" Jul 07 10:35:38.441248 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.441216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:38.443789 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.443760 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/abf3b900-88a7-4506-84e7-d708564acbca-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kv8xd\" (UID: \"abf3b900-88a7-4506-84e7-d708564acbca\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:38.679281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.679253 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" Jul 07 10:35:38.824297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:38.824274 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd"] Jul 07 10:35:38.826885 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:35:38.826859 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabf3b900_88a7_4506_84e7_d708564acbca.slice/crio-c707d41234d04fba62ec654805bd70ad5c7b641e7ee53d874bfa073d99e59492 WatchSource:0}: Error finding container c707d41234d04fba62ec654805bd70ad5c7b641e7ee53d874bfa073d99e59492: Status 404 returned error can't find the container with id c707d41234d04fba62ec654805bd70ad5c7b641e7ee53d874bfa073d99e59492 Jul 07 10:35:39.221767 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:39.221734 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" event={"ID":"abf3b900-88a7-4506-84e7-d708564acbca","Type":"ContainerStarted","Data":"c707d41234d04fba62ec654805bd70ad5c7b641e7ee53d874bfa073d99e59492"} Jul 07 10:35:40.693032 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:40.692984 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:40.858297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:40.858257 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:43.538192 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:43.538149 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:45.064331 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:45.064283 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:46.209718 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:46.209686 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:47.887947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:47.887901 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:47.988572 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:47.988539 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:48.112609 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.112571 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h"] Jul 07 10:35:48.112848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.112819 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" containerName="manager" containerID="cri-o://5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c" gracePeriod=2 Jul 07 10:35:48.119696 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.119666 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h"] Jul 07 10:35:48.120575 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.120538 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:48.144068 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.144029 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh"] Jul 07 10:35:48.144338 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.144281 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" containerName="manager" containerID="cri-o://46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0" gracePeriod=2 Jul 07 10:35:48.157910 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.157866 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:48.160127 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.160101 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh"] Jul 07 10:35:48.161155 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.161126 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:48.199487 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.199451 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:35:48.199994 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.199972 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" containerName="manager" Jul 07 10:35:48.199994 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.199993 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" containerName="manager" Jul 07 10:35:48.199994 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.200006 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" containerName="manager" Jul 07 10:35:48.200196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.200015 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" containerName="manager" Jul 07 10:35:48.200196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.200121 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" containerName="manager" Jul 07 10:35:48.200196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.200137 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" containerName="manager" Jul 07 10:35:48.203520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.203499 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.206774 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.206744 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:48.246174 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.245216 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:35:48.247285 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.247258 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt"] Jul 07 10:35:48.250941 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.250921 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:35:48.267481 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.267448 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt"] Jul 07 10:35:48.288116 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.288070 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:48.292279 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.292252 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:35:48.294598 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:35:48.294564 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:35:48.336494 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.336462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj6qw\" (UniqueName: \"kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.336687 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.336529 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrw68\" (UniqueName: \"kubernetes.io/projected/404d5b52-f916-44b0-bc74-558ad98ad138-kube-api-access-mrw68\") pod \"limitador-operator-controller-manager-859d7b78bd-4hdbt\" (UID: \"404d5b52-f916-44b0-bc74-558ad98ad138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:35:48.336687 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.336596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.437901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.437778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj6qw\" (UniqueName: \"kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.437901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.437865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrw68\" (UniqueName: \"kubernetes.io/projected/404d5b52-f916-44b0-bc74-558ad98ad138-kube-api-access-mrw68\") pod \"limitador-operator-controller-manager-859d7b78bd-4hdbt\" (UID: \"404d5b52-f916-44b0-bc74-558ad98ad138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:35:48.437901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.437886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.438281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.438264 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.456530 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.456493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrw68\" (UniqueName: \"kubernetes.io/projected/404d5b52-f916-44b0-bc74-558ad98ad138-kube-api-access-mrw68\") pod \"limitador-operator-controller-manager-859d7b78bd-4hdbt\" (UID: \"404d5b52-f916-44b0-bc74-558ad98ad138\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:35:48.456692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.456642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj6qw\" (UniqueName: \"kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gghv5\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.525590 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.525554 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:35:48.562879 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:48.562842 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:35:51.502873 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:51.502826 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:51.636955 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:51.636922 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:54.284765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:54.284708 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:54.357304 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:54.357265 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:58.468125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:58.468085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:58.883039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:58.882999 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:35:58.983814 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:35:58.983769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:00.634980 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.634950 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:00.637949 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:00.637911 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:00.656742 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.656716 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:00.659420 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:00.659390 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:00.661825 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:00.661774 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:00.677642 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.677617 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:36:00.679579 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:00.679552 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54bc10ae_ce4f_4a09_bf2a_1ac30b513c52.slice/crio-80679ceb2847a4c0f557160998c8f0880765c037aced40ad84d46db0a0f49df9 WatchSource:0}: Error finding container 80679ceb2847a4c0f557160998c8f0880765c037aced40ad84d46db0a0f49df9: Status 404 returned error can't find the container with id 80679ceb2847a4c0f557160998c8f0880765c037aced40ad84d46db0a0f49df9 Jul 07 10:36:00.700919 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.700894 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt"] Jul 07 10:36:00.716041 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:00.716017 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404d5b52_f916_44b0_bc74_558ad98ad138.slice/crio-defb0e55e94b034c7f52e08a951f239a8ac11bbdac1d64b80866b1f34d6be31b WatchSource:0}: Error finding container defb0e55e94b034c7f52e08a951f239a8ac11bbdac1d64b80866b1f34d6be31b: Status 404 returned error can't find the container with id defb0e55e94b034c7f52e08a951f239a8ac11bbdac1d64b80866b1f34d6be31b Jul 07 10:36:00.755670 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.755649 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume\") pod \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " Jul 07 10:36:00.755761 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.755686 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pknrv\" (UniqueName: \"kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv\") pod \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\" (UID: \"eeb8977a-127d-4f82-adc1-7bae2b74c62c\") " Jul 07 10:36:00.755837 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.755761 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7hd6\" (UniqueName: \"kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6\") pod \"e4818d64-7ee5-487b-8998-162de3ccdc6f\" (UID: \"e4818d64-7ee5-487b-8998-162de3ccdc6f\") " Jul 07 10:36:00.756221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.756199 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eeb8977a-127d-4f82-adc1-7bae2b74c62c" (UID: "eeb8977a-127d-4f82-adc1-7bae2b74c62c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:36:00.757975 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.757946 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv" (OuterVolumeSpecName: "kube-api-access-pknrv") pod "eeb8977a-127d-4f82-adc1-7bae2b74c62c" (UID: "eeb8977a-127d-4f82-adc1-7bae2b74c62c"). InnerVolumeSpecName "kube-api-access-pknrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:36:00.758092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.758068 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6" (OuterVolumeSpecName: "kube-api-access-q7hd6") pod "e4818d64-7ee5-487b-8998-162de3ccdc6f" (UID: "e4818d64-7ee5-487b-8998-162de3ccdc6f"). InnerVolumeSpecName "kube-api-access-q7hd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:36:00.857422 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.857390 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eeb8977a-127d-4f82-adc1-7bae2b74c62c-extensions-socket-volume\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:00.857422 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.857422 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pknrv\" (UniqueName: \"kubernetes.io/projected/eeb8977a-127d-4f82-adc1-7bae2b74c62c-kube-api-access-pknrv\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:00.857630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:00.857436 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7hd6\" (UniqueName: \"kubernetes.io/projected/e4818d64-7ee5-487b-8998-162de3ccdc6f-kube-api-access-q7hd6\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:01.080244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.080155 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" path="/var/lib/kubelet/pods/e4818d64-7ee5-487b-8998-162de3ccdc6f/volumes" Jul 07 10:36:01.080562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.080548 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" path="/var/lib/kubelet/pods/eeb8977a-127d-4f82-adc1-7bae2b74c62c/volumes" Jul 07 10:36:01.356328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.356237 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4818d64-7ee5-487b-8998-162de3ccdc6f" containerID="46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0" exitCode=0 Jul 07 10:36:01.356328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.356282 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:01.356575 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.356351 2581 scope.go:117] "RemoveContainer" containerID="46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0" Jul 07 10:36:01.357669 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.357644 2581 generic.go:356] "Generic (PLEG): container finished" podID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" containerID="5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c" exitCode=0 Jul 07 10:36:01.357790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.357687 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:01.359096 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.359058 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:01.359785 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.359710 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" event={"ID":"abf3b900-88a7-4506-84e7-d708564acbca","Type":"ContainerStarted","Data":"0f5cc388d1d589b9805560eb994552554445d6b7c1fa69109d766b513b69fc4f"} Jul 07 10:36:01.361559 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.361533 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" event={"ID":"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52","Type":"ContainerStarted","Data":"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91"} Jul 07 10:36:01.361676 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.361583 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" event={"ID":"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52","Type":"ContainerStarted","Data":"80679ceb2847a4c0f557160998c8f0880765c037aced40ad84d46db0a0f49df9"} Jul 07 10:36:01.361676 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.361583 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:01.361676 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.361605 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:36:01.363066 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.363046 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" event={"ID":"404d5b52-f916-44b0-bc74-558ad98ad138","Type":"ContainerStarted","Data":"ef06c88f5d56f1b6a1044e2bfd8a4fc0b8c14bcb422767fd2bce49b3286c456c"} Jul 07 10:36:01.363162 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.363080 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" event={"ID":"404d5b52-f916-44b0-bc74-558ad98ad138","Type":"ContainerStarted","Data":"defb0e55e94b034c7f52e08a951f239a8ac11bbdac1d64b80866b1f34d6be31b"} Jul 07 10:36:01.363215 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.363169 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:36:01.363785 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.363762 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:01.365765 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.365747 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:01.365902 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.365887 2581 scope.go:117] "RemoveContainer" containerID="46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0" Jul 07 10:36:01.366140 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.366121 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0\": container with ID starting with 46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0 not found: ID does not exist" containerID="46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0" Jul 07 10:36:01.366217 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.366148 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0"} err="failed to get container status \"46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0\": rpc error: code = NotFound desc = could not find container \"46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0\": container with ID starting with 46423c85e18005d3e2c33345b84bc031710341ce00844d7d2db0437722ebf3e0 not found: ID does not exist" Jul 07 10:36:01.366217 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.366164 2581 scope.go:117] "RemoveContainer" containerID="5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c" Jul 07 10:36:01.374763 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.374747 2581 scope.go:117] "RemoveContainer" containerID="5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c" Jul 07 10:36:01.375052 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.375032 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c\": container with ID starting with 5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c not found: ID does not exist" containerID="5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c" Jul 07 10:36:01.375102 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.375060 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c"} err="failed to get container status \"5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c\": rpc error: code = NotFound desc = could not find container \"5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c\": container with ID starting with 5fb8fa374fbedf80db9ca8b53647d855a036be1a638bb7015d642a349ddd536c not found: ID does not exist" Jul 07 10:36:01.383259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.383218 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kv8xd" podStartSLOduration=2.616279727 podStartE2EDuration="24.383204915s" totalImagesPullingTime="21.766925188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:01.382246224 +0000 UTC m=+528.842780982" watchObservedRunningTime="2026-07-07 10:36:01.383204915 +0000 UTC m=+528.843739659" Jul 07 10:36:01.404488 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.404432 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" podStartSLOduration=13.404412075 podStartE2EDuration="13.404412075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:01.402587999 +0000 UTC m=+528.863122789" watchObservedRunningTime="2026-07-07 10:36:01.404412075 +0000 UTC m=+528.864946822" Jul 07 10:36:01.424119 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:01.424066 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" podStartSLOduration=13.424050002 podStartE2EDuration="13.424050002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:01.422640001 +0000 UTC m=+528.883174784" watchObservedRunningTime="2026-07-07 10:36:01.424050002 +0000 UTC m=+528.884584747" Jul 07 10:36:01.425044 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:01.425020 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:03.078423 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:03.078377 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-sh4hh\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="e4818d64-7ee5-487b-8998-162de3ccdc6f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sh4hh" Jul 07 10:36:03.081195 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:03.081169 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xqj8h\" is forbidden: User \"system:node:ip-10-0-139-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-204.ec2.internal' and this object" podUID="eeb8977a-127d-4f82-adc1-7bae2b74c62c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xqj8h" Jul 07 10:36:05.109647 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.109566 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:36:05.110102 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.109760 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" podUID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" containerName="manager" containerID="cri-o://dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91" gracePeriod=10 Jul 07 10:36:05.111638 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.111618 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:36:05.358411 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.358374 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:36:05.391273 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.391239 2581 generic.go:356] "Generic (PLEG): container finished" podID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" containerID="dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91" exitCode=0 Jul 07 10:36:05.391453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.391325 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" event={"ID":"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52","Type":"ContainerDied","Data":"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91"} Jul 07 10:36:05.391453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.391377 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" event={"ID":"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52","Type":"ContainerDied","Data":"80679ceb2847a4c0f557160998c8f0880765c037aced40ad84d46db0a0f49df9"} Jul 07 10:36:05.391453 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.391391 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5" Jul 07 10:36:05.391624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.391403 2581 scope.go:117] "RemoveContainer" containerID="dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91" Jul 07 10:36:05.401942 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.401923 2581 scope.go:117] "RemoveContainer" containerID="dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91" Jul 07 10:36:05.402222 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:05.402201 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91\": container with ID starting with dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91 not found: ID does not exist" containerID="dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91" Jul 07 10:36:05.402280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.402230 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91"} err="failed to get container status \"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91\": rpc error: code = NotFound desc = could not find container \"dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91\": container with ID starting with dcb8492e4215a831623e5e77bf6281080ea4f75bfe7cc5fc13159fb4ce406b91 not found: ID does not exist" Jul 07 10:36:05.500680 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.500651 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume\") pod \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " Jul 07 10:36:05.500911 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.500709 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj6qw\" (UniqueName: \"kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw\") pod \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\" (UID: \"54bc10ae-ce4f-4a09-bf2a-1ac30b513c52\") " Jul 07 10:36:05.501105 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.501082 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" (UID: "54bc10ae-ce4f-4a09-bf2a-1ac30b513c52"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:36:05.502982 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.502955 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw" (OuterVolumeSpecName: "kube-api-access-tj6qw") pod "54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" (UID: "54bc10ae-ce4f-4a09-bf2a-1ac30b513c52"). InnerVolumeSpecName "kube-api-access-tj6qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:36:05.601575 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.601537 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-extensions-socket-volume\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:05.601575 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.601567 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj6qw\" (UniqueName: \"kubernetes.io/projected/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52-kube-api-access-tj6qw\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:05.723729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.723695 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:36:05.727743 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:05.727719 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gghv5"] Jul 07 10:36:07.079026 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:07.078991 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" path="/var/lib/kubelet/pods/54bc10ae-ce4f-4a09-bf2a-1ac30b513c52/volumes" Jul 07 10:36:08.867610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:08.867574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:12.377089 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:12.377058 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4hdbt" Jul 07 10:36:13.785632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:13.785601 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:13.900474 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:13.900434 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:20.854988 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:20.854946 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:21.677858 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.677827 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv"] Jul 07 10:36:21.678196 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.678184 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" containerName="manager" Jul 07 10:36:21.678243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.678199 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" containerName="manager" Jul 07 10:36:21.678276 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.678253 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="54bc10ae-ce4f-4a09-bf2a-1ac30b513c52" containerName="manager" Jul 07 10:36:21.698957 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.698916 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:21.808182 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.808146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:21.836571 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.836545 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv"] Jul 07 10:36:21.836713 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.836641 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.839366 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.839343 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wt57w\"" Jul 07 10:36:21.943916 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.943821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr75m\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-kube-api-access-hr75m\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.943916 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.943869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.943934 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.943978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.943996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.944047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.944100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.944170 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:21.944349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:21.944191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044616 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr75m\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-kube-api-access-hr75m\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.044791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.044952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.045052 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.045120 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.045196 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.045236 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.045727 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.045708 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.047245 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.047219 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.047533 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.047516 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.059681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.059659 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr75m\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-kube-api-access-hr75m\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.060251 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.060226 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4eaadc-8284-4fa1-8c0d-5a159484a916-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2nhsv\" (UID: \"3a4eaadc-8284-4fa1-8c0d-5a159484a916\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.146425 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.146389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:22.375057 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.375027 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv"] Jul 07 10:36:22.376945 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:22.376916 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4eaadc_8284_4fa1_8c0d_5a159484a916.slice/crio-32dd36fa9f403781513fa02c045591031fd413191c9d3b36d2a551d3753bf4ad WatchSource:0}: Error finding container 32dd36fa9f403781513fa02c045591031fd413191c9d3b36d2a551d3753bf4ad: Status 404 returned error can't find the container with id 32dd36fa9f403781513fa02c045591031fd413191c9d3b36d2a551d3753bf4ad Jul 07 10:36:22.379160 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.379131 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:36:22.379268 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.379206 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:36:22.379268 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.379248 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:36:22.480206 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:22.480175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" event={"ID":"3a4eaadc-8284-4fa1-8c0d-5a159484a916","Type":"ContainerStarted","Data":"32dd36fa9f403781513fa02c045591031fd413191c9d3b36d2a551d3753bf4ad"} Jul 07 10:36:23.486630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:23.486596 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" event={"ID":"3a4eaadc-8284-4fa1-8c0d-5a159484a916","Type":"ContainerStarted","Data":"450c6e19f332c518761275f5fc2baed9bddc909da153fb67b62e740933a8645e"} Jul 07 10:36:23.512170 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:23.512112 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podStartSLOduration=2.512089555 podStartE2EDuration="2.512089555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:23.510978945 +0000 UTC m=+550.971513708" watchObservedRunningTime="2026-07-07 10:36:23.512089555 +0000 UTC m=+550.972624303" Jul 07 10:36:24.147080 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.147046 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:24.152167 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.152138 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:24.491941 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.491867 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:24.493028 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.493013 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:36:24.554873 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.554839 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:24.560120 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.560090 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:24.631447 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.631416 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:24.631624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.631552 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.634254 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.634226 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 10:36:24.657615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.657585 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:24.668954 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.668928 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:24.670707 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.670684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.670860 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.670739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8xz8\" (UniqueName: \"kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.715525 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.715493 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:24.771863 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.771769 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.772046 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.771868 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8xz8\" (UniqueName: \"kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.772416 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.772396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.781600 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.781576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8xz8\" (UniqueName: \"kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8\") pod \"limitador-limitador-64c94fd64f-f5xg4\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:24.947034 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:24.946996 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:25.112379 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.112353 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:25.114441 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:25.114402 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2491b1_fa06_45fe_8537_41239b9f0148.slice/crio-55785ba3d4bf3e58126dcee9d37ff136ac43b5b0600de229ca2e833593bef124 WatchSource:0}: Error finding container 55785ba3d4bf3e58126dcee9d37ff136ac43b5b0600de229ca2e833593bef124: Status 404 returned error can't find the container with id 55785ba3d4bf3e58126dcee9d37ff136ac43b5b0600de229ca2e833593bef124 Jul 07 10:36:25.369091 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.369062 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:25.404179 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.404148 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:25.505940 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.505902 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:25.594057 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.594027 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:25.594057 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.594057 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" event={"ID":"fc2491b1-fa06-45fe-8537-41239b9f0148","Type":"ContainerStarted","Data":"55785ba3d4bf3e58126dcee9d37ff136ac43b5b0600de229ca2e833593bef124"} Jul 07 10:36:25.594250 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.594153 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:25.597038 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.597021 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dts5m\"" Jul 07 10:36:25.680968 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.680880 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltbl7\" (UniqueName: \"kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7\") pod \"authorino-58cd955675-d9fhl\" (UID: \"470d1851-4854-4a22-8268-15df91d84bdc\") " pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:25.781744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.781708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltbl7\" (UniqueName: \"kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7\") pod \"authorino-58cd955675-d9fhl\" (UID: \"470d1851-4854-4a22-8268-15df91d84bdc\") " pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:25.793485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.793459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltbl7\" (UniqueName: \"kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7\") pod \"authorino-58cd955675-d9fhl\" (UID: \"470d1851-4854-4a22-8268-15df91d84bdc\") " pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:25.904298 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:25.904270 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:26.125554 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:26.125531 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:26.127224 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:26.127107 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod470d1851_4854_4a22_8268_15df91d84bdc.slice/crio-e25223e62db6ddb6ff2f4558d86127a8dd5a42cd3fc9aeb5444f588675d78b0a WatchSource:0}: Error finding container e25223e62db6ddb6ff2f4558d86127a8dd5a42cd3fc9aeb5444f588675d78b0a: Status 404 returned error can't find the container with id e25223e62db6ddb6ff2f4558d86127a8dd5a42cd3fc9aeb5444f588675d78b0a Jul 07 10:36:26.509234 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:26.509171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-d9fhl" event={"ID":"470d1851-4854-4a22-8268-15df91d84bdc","Type":"ContainerStarted","Data":"e25223e62db6ddb6ff2f4558d86127a8dd5a42cd3fc9aeb5444f588675d78b0a"} Jul 07 10:36:29.543683 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:29.543642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:29.639958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:29.639923 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:30.375516 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:30.375478 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:31.539848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.539777 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" event={"ID":"fc2491b1-fa06-45fe-8537-41239b9f0148","Type":"ContainerStarted","Data":"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9"} Jul 07 10:36:31.540328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.540020 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:31.541422 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.541402 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-d9fhl" event={"ID":"470d1851-4854-4a22-8268-15df91d84bdc","Type":"ContainerStarted","Data":"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae"} Jul 07 10:36:31.541562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.541526 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-d9fhl" podUID="470d1851-4854-4a22-8268-15df91d84bdc" containerName="authorino" containerID="cri-o://d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae" gracePeriod=30 Jul 07 10:36:31.587002 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.586951 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" podStartSLOduration=1.6698091179999999 podStartE2EDuration="7.586933101s" totalImagesPullingTime="5.917123983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:31.58538411 +0000 UTC m=+559.045918855" watchObservedRunningTime="2026-07-07 10:36:31.586933101 +0000 UTC m=+559.047467845" Jul 07 10:36:31.620500 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.620441 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-d9fhl" podStartSLOduration=1.707507007 podStartE2EDuration="6.620424731s" totalImagesPullingTime="4.912917724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:31.616710399 +0000 UTC m=+559.077245143" watchObservedRunningTime="2026-07-07 10:36:31.620424731 +0000 UTC m=+559.080959476" Jul 07 10:36:31.793692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.793637 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:31.843756 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.843727 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltbl7\" (UniqueName: \"kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7\") pod \"470d1851-4854-4a22-8268-15df91d84bdc\" (UID: \"470d1851-4854-4a22-8268-15df91d84bdc\") " Jul 07 10:36:31.845897 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.845865 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7" (OuterVolumeSpecName: "kube-api-access-ltbl7") pod "470d1851-4854-4a22-8268-15df91d84bdc" (UID: "470d1851-4854-4a22-8268-15df91d84bdc"). InnerVolumeSpecName "kube-api-access-ltbl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:36:31.944574 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:31.944535 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltbl7\" (UniqueName: \"kubernetes.io/projected/470d1851-4854-4a22-8268-15df91d84bdc-kube-api-access-ltbl7\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:32.548154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.548119 2581 generic.go:356] "Generic (PLEG): container finished" podID="470d1851-4854-4a22-8268-15df91d84bdc" containerID="d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae" exitCode=0 Jul 07 10:36:32.548632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.548165 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-d9fhl" Jul 07 10:36:32.548632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.548199 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-d9fhl" event={"ID":"470d1851-4854-4a22-8268-15df91d84bdc","Type":"ContainerDied","Data":"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae"} Jul 07 10:36:32.548632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.548242 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-d9fhl" event={"ID":"470d1851-4854-4a22-8268-15df91d84bdc","Type":"ContainerDied","Data":"e25223e62db6ddb6ff2f4558d86127a8dd5a42cd3fc9aeb5444f588675d78b0a"} Jul 07 10:36:32.548632 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.548260 2581 scope.go:117] "RemoveContainer" containerID="d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae" Jul 07 10:36:32.557564 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.557547 2581 scope.go:117] "RemoveContainer" containerID="d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae" Jul 07 10:36:32.557867 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:32.557844 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae\": container with ID starting with d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae not found: ID does not exist" containerID="d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae" Jul 07 10:36:32.557955 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.557872 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae"} err="failed to get container status \"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae\": rpc error: code = NotFound desc = could not find container \"d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae\": container with ID starting with d6521917077ad5042a3e4d2083a7cf756c99f66bb214403bd9423d0b81f30bae not found: ID does not exist" Jul 07 10:36:32.572747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.572722 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:32.577008 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:32.576986 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-d9fhl"] Jul 07 10:36:33.079037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:33.079007 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="470d1851-4854-4a22-8268-15df91d84bdc" path="/var/lib/kubelet/pods/470d1851-4854-4a22-8268-15df91d84bdc/volumes" Jul 07 10:36:39.354503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:39.354471 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:39.354995 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:39.354653 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" podUID="fc2491b1-fa06-45fe-8537-41239b9f0148" containerName="limitador" containerID="cri-o://443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9" gracePeriod=30 Jul 07 10:36:39.357195 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:39.357165 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:39.911856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:39.911830 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:40.018788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.018702 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file\") pod \"fc2491b1-fa06-45fe-8537-41239b9f0148\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " Jul 07 10:36:40.018788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.018772 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8xz8\" (UniqueName: \"kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8\") pod \"fc2491b1-fa06-45fe-8537-41239b9f0148\" (UID: \"fc2491b1-fa06-45fe-8537-41239b9f0148\") " Jul 07 10:36:40.019089 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.019069 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file" (OuterVolumeSpecName: "config-file") pod "fc2491b1-fa06-45fe-8537-41239b9f0148" (UID: "fc2491b1-fa06-45fe-8537-41239b9f0148"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:36:40.021029 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.021008 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8" (OuterVolumeSpecName: "kube-api-access-h8xz8") pod "fc2491b1-fa06-45fe-8537-41239b9f0148" (UID: "fc2491b1-fa06-45fe-8537-41239b9f0148"). InnerVolumeSpecName "kube-api-access-h8xz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:36:40.119590 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.119552 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fc2491b1-fa06-45fe-8537-41239b9f0148-config-file\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:40.119590 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.119585 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8xz8\" (UniqueName: \"kubernetes.io/projected/fc2491b1-fa06-45fe-8537-41239b9f0148-kube-api-access-h8xz8\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:36:40.586517 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.586485 2581 generic.go:356] "Generic (PLEG): container finished" podID="fc2491b1-fa06-45fe-8537-41239b9f0148" containerID="443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9" exitCode=0 Jul 07 10:36:40.587141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.586528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" event={"ID":"fc2491b1-fa06-45fe-8537-41239b9f0148","Type":"ContainerDied","Data":"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9"} Jul 07 10:36:40.587141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.586536 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" Jul 07 10:36:40.587141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.586568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-f5xg4" event={"ID":"fc2491b1-fa06-45fe-8537-41239b9f0148","Type":"ContainerDied","Data":"55785ba3d4bf3e58126dcee9d37ff136ac43b5b0600de229ca2e833593bef124"} Jul 07 10:36:40.587141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.586589 2581 scope.go:117] "RemoveContainer" containerID="443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9" Jul 07 10:36:40.595345 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.595327 2581 scope.go:117] "RemoveContainer" containerID="443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9" Jul 07 10:36:40.595627 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:36:40.595606 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9\": container with ID starting with 443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9 not found: ID does not exist" containerID="443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9" Jul 07 10:36:40.595672 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.595637 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9"} err="failed to get container status \"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9\": rpc error: code = NotFound desc = could not find container \"443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9\": container with ID starting with 443c852177dcb3ff7522b135809918f28f67b0a48155b5817385bc377721f6b9 not found: ID does not exist" Jul 07 10:36:40.609201 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.609176 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:40.618496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:40.616331 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-f5xg4"] Jul 07 10:36:41.043513 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.043472 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-8tjtm"] Jul 07 10:36:41.043975 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.043956 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470d1851-4854-4a22-8268-15df91d84bdc" containerName="authorino" Jul 07 10:36:41.044076 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.043977 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="470d1851-4854-4a22-8268-15df91d84bdc" containerName="authorino" Jul 07 10:36:41.044076 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.044006 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc2491b1-fa06-45fe-8537-41239b9f0148" containerName="limitador" Jul 07 10:36:41.044076 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.044015 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2491b1-fa06-45fe-8537-41239b9f0148" containerName="limitador" Jul 07 10:36:41.044221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.044116 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc2491b1-fa06-45fe-8537-41239b9f0148" containerName="limitador" Jul 07 10:36:41.044221 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.044129 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="470d1851-4854-4a22-8268-15df91d84bdc" containerName="authorino" Jul 07 10:36:41.048853 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.048833 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.052356 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.052336 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 10:36:41.052462 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.052338 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cg5kf\"" Jul 07 10:36:41.057824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.057781 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8tjtm"] Jul 07 10:36:41.079536 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.079504 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2491b1-fa06-45fe-8537-41239b9f0148" path="/var/lib/kubelet/pods/fc2491b1-fa06-45fe-8537-41239b9f0148/volumes" Jul 07 10:36:41.089282 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.089257 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:41.128305 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.128279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e78edd62-7446-4726-8ba6-7236de296c87-data\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.128407 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.128320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fldgp\" (UniqueName: \"kubernetes.io/projected/e78edd62-7446-4726-8ba6-7236de296c87-kube-api-access-fldgp\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.229412 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.229382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e78edd62-7446-4726-8ba6-7236de296c87-data\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.229592 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.229445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fldgp\" (UniqueName: \"kubernetes.io/projected/e78edd62-7446-4726-8ba6-7236de296c87-kube-api-access-fldgp\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.229834 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.229791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e78edd62-7446-4726-8ba6-7236de296c87-data\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.240501 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.240468 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fldgp\" (UniqueName: \"kubernetes.io/projected/e78edd62-7446-4726-8ba6-7236de296c87-kube-api-access-fldgp\") pod \"postgres-69fb4594cf-8tjtm\" (UID: \"e78edd62-7446-4726-8ba6-7236de296c87\") " pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.360541 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.360444 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:41.515586 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.515535 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8tjtm"] Jul 07 10:36:41.518473 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:41.518447 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78edd62_7446_4726_8ba6_7236de296c87.slice/crio-9be0039be5aa66477163dc9daf4f0efa6222cb8063d33fe27fa03d6f9f1256e7 WatchSource:0}: Error finding container 9be0039be5aa66477163dc9daf4f0efa6222cb8063d33fe27fa03d6f9f1256e7: Status 404 returned error can't find the container with id 9be0039be5aa66477163dc9daf4f0efa6222cb8063d33fe27fa03d6f9f1256e7 Jul 07 10:36:41.592513 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:41.592478 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8tjtm" event={"ID":"e78edd62-7446-4726-8ba6-7236de296c87","Type":"ContainerStarted","Data":"9be0039be5aa66477163dc9daf4f0efa6222cb8063d33fe27fa03d6f9f1256e7"} Jul 07 10:36:46.623025 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:46.622995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8tjtm" event={"ID":"e78edd62-7446-4726-8ba6-7236de296c87","Type":"ContainerStarted","Data":"2a65cd1dfed6c2ffede7a126e05fe512bbfc7edaf4149bdfbc58916dc0890282"} Jul 07 10:36:46.623489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:46.623051 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:46.655352 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:46.655300 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-8tjtm" podStartSLOduration=0.949332778 podStartE2EDuration="5.655285494s" totalImagesPullingTime="4.705952716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:46.654191823 +0000 UTC m=+574.114726567" watchObservedRunningTime="2026-07-07 10:36:46.655285494 +0000 UTC m=+574.115820239" Jul 07 10:36:52.658675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:52.658641 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-8tjtm" Jul 07 10:36:53.059240 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.059117 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:53.179672 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.179643 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:36:53.682922 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.682886 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:36:53.689202 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.689170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.692427 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.692406 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 10:36:53.692834 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.692782 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dts5m\"" Jul 07 10:36:53.699487 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.699465 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:36:53.851114 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.851077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7l54\" (UniqueName: \"kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.851114 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.851117 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.952571 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.952477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7l54\" (UniqueName: \"kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.952571 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.952522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.955179 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.955154 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.967346 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.967317 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7l54\" (UniqueName: \"kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54\") pod \"authorino-6c76ddf4bf-9sdhv\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:53.998088 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:53.998053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:36:54.138862 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:54.138836 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:36:54.141096 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:36:54.141064 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf17c1577_7353_44ba_ab30_d51405d36ac4.slice/crio-152f8d7385bc0ec46a0b554b8ec38ea06a404749361088bebaf0ee790a3ac6ef WatchSource:0}: Error finding container 152f8d7385bc0ec46a0b554b8ec38ea06a404749361088bebaf0ee790a3ac6ef: Status 404 returned error can't find the container with id 152f8d7385bc0ec46a0b554b8ec38ea06a404749361088bebaf0ee790a3ac6ef Jul 07 10:36:54.663305 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:54.663271 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" event={"ID":"f17c1577-7353-44ba-ab30-d51405d36ac4","Type":"ContainerStarted","Data":"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb"} Jul 07 10:36:54.663493 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:54.663314 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" event={"ID":"f17c1577-7353-44ba-ab30-d51405d36ac4","Type":"ContainerStarted","Data":"152f8d7385bc0ec46a0b554b8ec38ea06a404749361088bebaf0ee790a3ac6ef"} Jul 07 10:36:54.698255 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:54.698155 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" podStartSLOduration=1.424535262 podStartE2EDuration="1.698136453s" totalImagesPullingTime="273.601191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:36:54.697023647 +0000 UTC m=+582.157558393" watchObservedRunningTime="2026-07-07 10:36:54.698136453 +0000 UTC m=+582.158671198" Jul 07 10:36:59.848280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:36:59.848233 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:00.410400 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.410366 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:00.563076 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.563041 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:37:00.639814 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.639777 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:37:00.640017 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.639917 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.644817 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.644732 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 10:37:00.644972 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.644875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:00.645604 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.645586 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ctqxp\"" Jul 07 10:37:00.645688 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.645645 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 10:37:00.811812 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.811707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4dmb\" (UniqueName: \"kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.811812 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.811781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.912486 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.912447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4dmb\" (UniqueName: \"kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.912924 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.912557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.915095 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.915075 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.932338 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.932318 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4dmb\" (UniqueName: \"kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb\") pod \"maas-controller-5bcd9f7d5c-f8tq7\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:00.952186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:00.952159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:01.158618 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:01.158587 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:37:01.161111 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:37:01.161084 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555ef1b2_e232_4169_818b_7fd2d8d18f4d.slice/crio-48d4daa22e1696a69007e5eaed5f17a40d2b13987305878fdda6129891c3e248 WatchSource:0}: Error finding container 48d4daa22e1696a69007e5eaed5f17a40d2b13987305878fdda6129891c3e248: Status 404 returned error can't find the container with id 48d4daa22e1696a69007e5eaed5f17a40d2b13987305878fdda6129891c3e248 Jul 07 10:37:01.702882 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:01.702845 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" event={"ID":"555ef1b2-e232-4169-818b-7fd2d8d18f4d","Type":"ContainerStarted","Data":"48d4daa22e1696a69007e5eaed5f17a40d2b13987305878fdda6129891c3e248"} Jul 07 10:37:02.846311 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:02.846282 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:03.201587 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:03.201553 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:03.715073 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:03.715037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" event={"ID":"555ef1b2-e232-4169-818b-7fd2d8d18f4d","Type":"ContainerStarted","Data":"eb31c162f9d9d6746bb63456c06012835a2f5f35c989db557fe0edcdcca8dbeb"} Jul 07 10:37:03.715246 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:03.715125 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:03.781362 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:03.781304 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" podStartSLOduration=2.100164193 podStartE2EDuration="3.781286208s" totalImagesPullingTime="1.681122015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:37:03.780305637 +0000 UTC m=+591.240840402" watchObservedRunningTime="2026-07-07 10:37:03.781286208 +0000 UTC m=+591.241820953" Jul 07 10:37:14.727521 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:14.727486 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:37:32.412929 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:32.412879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:32.963599 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:32.963559 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:33.058074 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.058045 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7fdd849877-9kxz9"] Jul 07 10:37:33.062348 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.062326 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.066690 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.066667 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qlm6h\"" Jul 07 10:37:33.067816 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.067786 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 10:37:33.092727 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.092700 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7fdd849877-9kxz9"] Jul 07 10:37:33.137637 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.137600 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:33.210983 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.210942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6768bfd-1a7e-49a1-853b-4140138e2ade-openshift-service-ca\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.211168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.210991 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2cnf\" (UniqueName: \"kubernetes.io/projected/a6768bfd-1a7e-49a1-853b-4140138e2ade-kube-api-access-m2cnf\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.211168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.211018 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6768bfd-1a7e-49a1-853b-4140138e2ade-maas-api-tls\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.312150 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.312065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6768bfd-1a7e-49a1-853b-4140138e2ade-openshift-service-ca\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.312150 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.312108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2cnf\" (UniqueName: \"kubernetes.io/projected/a6768bfd-1a7e-49a1-853b-4140138e2ade-kube-api-access-m2cnf\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.312150 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.312130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6768bfd-1a7e-49a1-853b-4140138e2ade-maas-api-tls\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.312764 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.312745 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6768bfd-1a7e-49a1-853b-4140138e2ade-openshift-service-ca\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.314759 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.314736 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6768bfd-1a7e-49a1-853b-4140138e2ade-maas-api-tls\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.324271 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.324246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2cnf\" (UniqueName: \"kubernetes.io/projected/a6768bfd-1a7e-49a1-853b-4140138e2ade-kube-api-access-m2cnf\") pod \"maas-api-7fdd849877-9kxz9\" (UID: \"a6768bfd-1a7e-49a1-853b-4140138e2ade\") " pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.372597 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.372561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:33.531193 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.531167 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7fdd849877-9kxz9"] Jul 07 10:37:33.533215 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:37:33.533179 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6768bfd_1a7e_49a1_853b_4140138e2ade.slice/crio-e840e0f8874f1e848e299725b3c8141daa821763161579769a4bf2e1c194b70f WatchSource:0}: Error finding container e840e0f8874f1e848e299725b3c8141daa821763161579769a4bf2e1c194b70f: Status 404 returned error can't find the container with id e840e0f8874f1e848e299725b3c8141daa821763161579769a4bf2e1c194b70f Jul 07 10:37:33.698782 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.698742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:33.814014 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.813920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:33.867827 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.867781 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7fdd849877-9kxz9" event={"ID":"a6768bfd-1a7e-49a1-853b-4140138e2ade","Type":"ContainerStarted","Data":"e840e0f8874f1e848e299725b3c8141daa821763161579769a4bf2e1c194b70f"} Jul 07 10:37:33.937871 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.937830 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fkfld"] Jul 07 10:37:33.944075 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.944054 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:33.949722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.949669 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rd2tb\"" Jul 07 10:37:33.949722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.949710 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 10:37:33.966499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:33.966471 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fkfld"] Jul 07 10:37:34.121516 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.121484 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2f24\" (UniqueName: \"kubernetes.io/projected/30b658d7-ce36-48e9-a904-a8cc598deb70-kube-api-access-n2f24\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.121745 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.121536 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30b658d7-ce36-48e9-a904-a8cc598deb70-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.222349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.222255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2f24\" (UniqueName: \"kubernetes.io/projected/30b658d7-ce36-48e9-a904-a8cc598deb70-kube-api-access-n2f24\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.222349 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.222345 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30b658d7-ce36-48e9-a904-a8cc598deb70-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.223182 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.223158 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30b658d7-ce36-48e9-a904-a8cc598deb70-plugins-config-volume\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.234362 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.234330 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2f24\" (UniqueName: \"kubernetes.io/projected/30b658d7-ce36-48e9-a904-a8cc598deb70-kube-api-access-n2f24\") pod \"payload-pre-processing-7964457696-fkfld\" (UID: \"30b658d7-ce36-48e9-a904-a8cc598deb70\") " pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.254243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.254208 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:34.437017 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:37:34.432397 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b658d7_ce36_48e9_a904_a8cc598deb70.slice/crio-f4ba97b97bf9bf5e5894988833b7f67d35ed562e7bee2722c13cacef1e056239 WatchSource:0}: Error finding container f4ba97b97bf9bf5e5894988833b7f67d35ed562e7bee2722c13cacef1e056239: Status 404 returned error can't find the container with id f4ba97b97bf9bf5e5894988833b7f67d35ed562e7bee2722c13cacef1e056239 Jul 07 10:37:34.437017 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.433879 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-fkfld"] Jul 07 10:37:34.875355 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:34.875291 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" event={"ID":"30b658d7-ce36-48e9-a904-a8cc598deb70","Type":"ContainerStarted","Data":"f4ba97b97bf9bf5e5894988833b7f67d35ed562e7bee2722c13cacef1e056239"} Jul 07 10:37:35.886015 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:35.885970 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7fdd849877-9kxz9" event={"ID":"a6768bfd-1a7e-49a1-853b-4140138e2ade","Type":"ContainerStarted","Data":"63e112726fc60a5332ec6b1d3fd7d595d768747f5ab9a3edb32093625d98c3f4"} Jul 07 10:37:35.886570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:35.886547 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:35.913050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:35.912994 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7fdd849877-9kxz9" podStartSLOduration=1.203167928 podStartE2EDuration="2.912973587s" totalImagesPullingTime="1.709805659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:37:35.911448865 +0000 UTC m=+623.371983638" watchObservedRunningTime="2026-07-07 10:37:35.912973587 +0000 UTC m=+623.373508331" Jul 07 10:37:36.893152 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:36.893113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" event={"ID":"30b658d7-ce36-48e9-a904-a8cc598deb70","Type":"ContainerStarted","Data":"790611acbd990ec0650d83e494bf70fc4bcc333894e8c3a740fa499e02d8b193"} Jul 07 10:37:36.893553 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:36.893377 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:36.912431 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:36.912386 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" podStartSLOduration=2.155813577 podStartE2EDuration="3.912369958s" totalImagesPullingTime="1.756556381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:37:36.911751664 +0000 UTC m=+624.372286430" watchObservedRunningTime="2026-07-07 10:37:36.912369958 +0000 UTC m=+624.372904702" Jul 07 10:37:40.533941 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:40.533902 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 10:37:40.534331 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:40.533956 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 10:37:41.898914 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.898882 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7fdd849877-9kxz9" Jul 07 10:37:41.921476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.921446 2581 generic.go:356] "Generic (PLEG): container finished" podID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerID="450c6e19f332c518761275f5fc2baed9bddc909da153fb67b62e740933a8645e" exitCode=0 Jul 07 10:37:41.921634 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.921524 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" event={"ID":"3a4eaadc-8284-4fa1-8c0d-5a159484a916","Type":"ContainerDied","Data":"450c6e19f332c518761275f5fc2baed9bddc909da153fb67b62e740933a8645e"} Jul 07 10:37:41.921922 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.921907 2581 scope.go:117] "RemoveContainer" containerID="450c6e19f332c518761275f5fc2baed9bddc909da153fb67b62e740933a8645e" Jul 07 10:37:41.922994 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.922945 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:37:41.923067 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.923050 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:37:41.923108 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:41.923096 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:37:42.928870 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:42.928834 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" event={"ID":"3a4eaadc-8284-4fa1-8c0d-5a159484a916","Type":"ContainerStarted","Data":"2a8e90e2a845a56b40800b25da5a53bc5ebf032fb4df9bfcc2de3554f7b9f651"} Jul 07 10:37:43.025960 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:43.025916 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:43.146670 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:43.146634 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:37:43.146943 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:43.146917 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 10:37:43.147071 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:43.146974 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 10:37:44.147059 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:44.147023 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 10:37:44.147515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:44.147095 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 10:37:44.809955 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:44.809928 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:45.147313 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:45.147275 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 10:37:45.147708 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:45.147328 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 10:37:46.147859 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:46.147790 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 10:37:46.148327 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:46.147884 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 10:37:47.146927 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:47.146890 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2nhsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 10:37:47.147110 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:47.146959 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" podUID="3a4eaadc-8284-4fa1-8c0d-5a159484a916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 10:37:47.900653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:47.900623 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-fkfld" Jul 07 10:37:48.150819 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:48.150724 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:37:48.151044 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:48.151025 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:37:48.151708 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:48.151690 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2nhsv" Jul 07 10:37:48.717710 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:48.717670 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:54.090715 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.090678 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96"] Jul 07 10:37:54.095406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.095390 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.099609 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.099585 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6qgk6\"" Jul 07 10:37:54.099609 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.099596 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 10:37:54.099811 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.099600 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 10:37:54.099811 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.099640 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 10:37:54.105112 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.105087 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96"] Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108496 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xzx\" (UniqueName: \"kubernetes.io/projected/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-kube-api-access-f8xzx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108538 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108574 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.110315 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.108707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.197865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.197828 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:37:54.210012 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.209987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210076 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xzx\" (UniqueName: \"kubernetes.io/projected/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-kube-api-access-f8xzx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210100 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210618 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210590 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210751 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210632 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.210751 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.210650 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.212633 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.212605 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.212744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.212638 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.219985 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.219961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xzx\" (UniqueName: \"kubernetes.io/projected/5c0aa737-1413-40a9-ae6d-343da9c2c6a9-kube-api-access-f8xzx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96\" (UID: \"5c0aa737-1413-40a9-ae6d-343da9c2c6a9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.405668 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.405637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:37:54.556441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.556405 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96"] Jul 07 10:37:54.560827 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:37:54.560762 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0aa737_1413_40a9_ae6d_343da9c2c6a9.slice/crio-cfceb38560c8afa9a8520ea3528fe3e72786953ea7f56ca13c3c08c9c8e70546 WatchSource:0}: Error finding container cfceb38560c8afa9a8520ea3528fe3e72786953ea7f56ca13c3c08c9c8e70546: Status 404 returned error can't find the container with id cfceb38560c8afa9a8520ea3528fe3e72786953ea7f56ca13c3c08c9c8e70546 Jul 07 10:37:54.562601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.562583 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 10:37:54.996219 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:54.996179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" event={"ID":"5c0aa737-1413-40a9-ae6d-343da9c2c6a9","Type":"ContainerStarted","Data":"cfceb38560c8afa9a8520ea3528fe3e72786953ea7f56ca13c3c08c9c8e70546"} Jul 07 10:37:56.003223 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:56.003113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" event={"ID":"5c0aa737-1413-40a9-ae6d-343da9c2c6a9","Type":"ContainerStarted","Data":"cecd2b85f80ece4371fc00748c1c4aec24be62e2d2ced83a165bd9864342c1c4"} Jul 07 10:37:56.025508 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:37:56.025447 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" podStartSLOduration=0.836315636 podStartE2EDuration="2.025430144s" totalImagesPullingTime="1.189114508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:37:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:37:56.023783151 +0000 UTC m=+643.484317936" watchObservedRunningTime="2026-07-07 10:37:56.025430144 +0000 UTC m=+643.485964891" Jul 07 10:38:04.183719 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:04.183671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:38:04.406699 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:04.406664 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:38:04.406943 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:04.406731 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:38:04.419585 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:04.419557 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:38:05.061467 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:05.061434 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96" Jul 07 10:38:16.903586 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:16.903550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:38:20.779574 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.779536 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5"] Jul 07 10:38:20.786500 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.786467 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.789629 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.789589 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 10:38:20.794991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.794710 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5"] Jul 07 10:38:20.821546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.821515 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:38:20.880521 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.880681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.880681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af42eec7-1175-4250-b829-2430c308c0e7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.880681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxchx\" (UniqueName: \"kubernetes.io/projected/af42eec7-1175-4250-b829-2430c308c0e7-kube-api-access-jxchx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.880681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.880681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.880667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982077 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982077 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982078 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af42eec7-1175-4250-b829-2430c308c0e7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxchx\" (UniqueName: \"kubernetes.io/projected/af42eec7-1175-4250-b829-2430c308c0e7-kube-api-access-jxchx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982491 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982377 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982491 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982440 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982720 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982693 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.982869 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.982761 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.984607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.984586 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af42eec7-1175-4250-b829-2430c308c0e7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.984781 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.984764 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af42eec7-1175-4250-b829-2430c308c0e7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:20.991251 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:20.991230 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxchx\" (UniqueName: \"kubernetes.io/projected/af42eec7-1175-4250-b829-2430c308c0e7-kube-api-access-jxchx\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5\" (UID: \"af42eec7-1175-4250-b829-2430c308c0e7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:21.099387 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:21.099305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:21.245243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:21.245218 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5"] Jul 07 10:38:21.246993 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:38:21.246959 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf42eec7_1175_4250_b829_2430c308c0e7.slice/crio-f760496e3d82adfce236bffddf35a2226944754ee7dfb1bb169ee968fbedaf38 WatchSource:0}: Error finding container f760496e3d82adfce236bffddf35a2226944754ee7dfb1bb169ee968fbedaf38: Status 404 returned error can't find the container with id f760496e3d82adfce236bffddf35a2226944754ee7dfb1bb169ee968fbedaf38 Jul 07 10:38:22.133855 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:22.133817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" event={"ID":"af42eec7-1175-4250-b829-2430c308c0e7","Type":"ContainerStarted","Data":"e1589d26383c1bb879cb6c8b3526fd8f075f023f1ecd9f2cdd7b1c7a29310e38"} Jul 07 10:38:22.134263 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:22.133880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" event={"ID":"af42eec7-1175-4250-b829-2430c308c0e7","Type":"ContainerStarted","Data":"f760496e3d82adfce236bffddf35a2226944754ee7dfb1bb169ee968fbedaf38"} Jul 07 10:38:22.154559 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:22.154510 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" podStartSLOduration=1.9693138399999999 podStartE2EDuration="2.154492677s" totalImagesPullingTime="185.178837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:38:22.15305233 +0000 UTC m=+669.613587101" watchObservedRunningTime="2026-07-07 10:38:22.154492677 +0000 UTC m=+669.615027422" Jul 07 10:38:27.627397 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:27.627337 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:38:31.099769 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:31.099734 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:31.100216 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:31.100079 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:31.112595 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:31.112573 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:31.191344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:31.191311 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5" Jul 07 10:38:45.230971 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:38:45.230935 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:39:33.457585 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.457502 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5797c5c47b-pfvm6"] Jul 07 10:39:33.461304 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.461282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.473394 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.473368 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5797c5c47b-pfvm6"] Jul 07 10:39:33.562679 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.562642 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/513172af-ef5f-47b1-a985-6130e64a6451-tls-cert\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.562906 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.562750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5xtr\" (UniqueName: \"kubernetes.io/projected/513172af-ef5f-47b1-a985-6130e64a6451-kube-api-access-w5xtr\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.663673 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.663646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/513172af-ef5f-47b1-a985-6130e64a6451-tls-cert\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.663870 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.663725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5xtr\" (UniqueName: \"kubernetes.io/projected/513172af-ef5f-47b1-a985-6130e64a6451-kube-api-access-w5xtr\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.666365 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.666332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/513172af-ef5f-47b1-a985-6130e64a6451-tls-cert\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.673029 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.673007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5xtr\" (UniqueName: \"kubernetes.io/projected/513172af-ef5f-47b1-a985-6130e64a6451-kube-api-access-w5xtr\") pod \"authorino-5797c5c47b-pfvm6\" (UID: \"513172af-ef5f-47b1-a985-6130e64a6451\") " pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.770955 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.770860 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5797c5c47b-pfvm6" Jul 07 10:39:33.824222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.824179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:39:33.925141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.925048 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5797c5c47b-pfvm6"] Jul 07 10:39:33.925428 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:33.925396 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:39:33.928148 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:39:33.928123 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod513172af_ef5f_47b1_a985_6130e64a6451.slice/crio-88943345e88d8f62eb4e883d2d398eaf604b29850cb839a0af312d66e3c67c10 WatchSource:0}: Error finding container 88943345e88d8f62eb4e883d2d398eaf604b29850cb839a0af312d66e3c67c10: Status 404 returned error can't find the container with id 88943345e88d8f62eb4e883d2d398eaf604b29850cb839a0af312d66e3c67c10 Jul 07 10:39:34.498602 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.498506 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5797c5c47b-pfvm6" event={"ID":"513172af-ef5f-47b1-a985-6130e64a6451","Type":"ContainerStarted","Data":"317e773ca5ac395ab5ec10fb759e9dfc4348c4e64936704c3bc0bb4ba45d511e"} Jul 07 10:39:34.498602 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.498564 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5797c5c47b-pfvm6" event={"ID":"513172af-ef5f-47b1-a985-6130e64a6451","Type":"ContainerStarted","Data":"88943345e88d8f62eb4e883d2d398eaf604b29850cb839a0af312d66e3c67c10"} Jul 07 10:39:34.611748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.611696 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5797c5c47b-pfvm6" podStartSLOduration=1.299631121 podStartE2EDuration="1.611678184s" totalImagesPullingTime="312.047063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:39:34.540154286 +0000 UTC m=+742.000689032" watchObservedRunningTime="2026-07-07 10:39:34.611678184 +0000 UTC m=+742.072212928" Jul 07 10:39:34.614908 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.614873 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:39:34.615094 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.615071 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" podUID="f17c1577-7353-44ba-ab30-d51405d36ac4" containerName="authorino" containerID="cri-o://1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb" gracePeriod=30 Jul 07 10:39:34.883192 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.883166 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:39:34.976323 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.976293 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert\") pod \"f17c1577-7353-44ba-ab30-d51405d36ac4\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " Jul 07 10:39:34.976514 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.976372 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7l54\" (UniqueName: \"kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54\") pod \"f17c1577-7353-44ba-ab30-d51405d36ac4\" (UID: \"f17c1577-7353-44ba-ab30-d51405d36ac4\") " Jul 07 10:39:34.978494 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.978468 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54" (OuterVolumeSpecName: "kube-api-access-j7l54") pod "f17c1577-7353-44ba-ab30-d51405d36ac4" (UID: "f17c1577-7353-44ba-ab30-d51405d36ac4"). InnerVolumeSpecName "kube-api-access-j7l54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:39:34.989855 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:34.989823 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f17c1577-7353-44ba-ab30-d51405d36ac4" (UID: "f17c1577-7353-44ba-ab30-d51405d36ac4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:39:35.077901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.077812 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f17c1577-7353-44ba-ab30-d51405d36ac4-tls-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:39:35.077901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.077845 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7l54\" (UniqueName: \"kubernetes.io/projected/f17c1577-7353-44ba-ab30-d51405d36ac4-kube-api-access-j7l54\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:39:35.111145 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.111109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:39:35.266355 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.266303 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:39:35.505639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.505597 2581 generic.go:356] "Generic (PLEG): container finished" podID="f17c1577-7353-44ba-ab30-d51405d36ac4" containerID="1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb" exitCode=0 Jul 07 10:39:35.505639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.505645 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" Jul 07 10:39:35.506266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.505680 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" event={"ID":"f17c1577-7353-44ba-ab30-d51405d36ac4","Type":"ContainerDied","Data":"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb"} Jul 07 10:39:35.506266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.505727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c76ddf4bf-9sdhv" event={"ID":"f17c1577-7353-44ba-ab30-d51405d36ac4","Type":"ContainerDied","Data":"152f8d7385bc0ec46a0b554b8ec38ea06a404749361088bebaf0ee790a3ac6ef"} Jul 07 10:39:35.506266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.505745 2581 scope.go:117] "RemoveContainer" containerID="1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb" Jul 07 10:39:35.515266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.515240 2581 scope.go:117] "RemoveContainer" containerID="1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb" Jul 07 10:39:35.515574 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:39:35.515553 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb\": container with ID starting with 1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb not found: ID does not exist" containerID="1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb" Jul 07 10:39:35.515645 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.515581 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb"} err="failed to get container status \"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb\": rpc error: code = NotFound desc = could not find container \"1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb\": container with ID starting with 1ac8f5418e7c3b2634d707e789c4426f8e9ef4347eb30bac746bf093e8e805eb not found: ID does not exist" Jul 07 10:39:35.545675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.545638 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:39:35.557382 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:35.557351 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c76ddf4bf-9sdhv"] Jul 07 10:39:37.079963 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:39:37.079929 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f17c1577-7353-44ba-ab30-d51405d36ac4" path="/var/lib/kubelet/pods/f17c1577-7353-44ba-ab30-d51405d36ac4/volumes" Jul 07 10:40:46.557911 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.557819 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:40:46.656416 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.656381 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:40:46.656674 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.656643 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" podUID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" containerName="manager" containerID="cri-o://eb31c162f9d9d6746bb63456c06012835a2f5f35c989db557fe0edcdcca8dbeb" gracePeriod=30 Jul 07 10:40:46.874306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.874277 2581 generic.go:356] "Generic (PLEG): container finished" podID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" containerID="eb31c162f9d9d6746bb63456c06012835a2f5f35c989db557fe0edcdcca8dbeb" exitCode=0 Jul 07 10:40:46.874499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.874346 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" event={"ID":"555ef1b2-e232-4169-818b-7fd2d8d18f4d","Type":"ContainerDied","Data":"eb31c162f9d9d6746bb63456c06012835a2f5f35c989db557fe0edcdcca8dbeb"} Jul 07 10:40:46.914542 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.914515 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:40:46.967005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.966974 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert\") pod \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " Jul 07 10:40:46.967191 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.967017 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4dmb\" (UniqueName: \"kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb\") pod \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\" (UID: \"555ef1b2-e232-4169-818b-7fd2d8d18f4d\") " Jul 07 10:40:46.969370 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.969342 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "555ef1b2-e232-4169-818b-7fd2d8d18f4d" (UID: "555ef1b2-e232-4169-818b-7fd2d8d18f4d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:40:46.969481 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:46.969421 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb" (OuterVolumeSpecName: "kube-api-access-b4dmb") pod "555ef1b2-e232-4169-818b-7fd2d8d18f4d" (UID: "555ef1b2-e232-4169-818b-7fd2d8d18f4d"). InnerVolumeSpecName "kube-api-access-b4dmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:40:47.068785 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.068747 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/555ef1b2-e232-4169-818b-7fd2d8d18f4d-webhook-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:40:47.068785 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.068780 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4dmb\" (UniqueName: \"kubernetes.io/projected/555ef1b2-e232-4169-818b-7fd2d8d18f4d-kube-api-access-b4dmb\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:40:47.880376 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.880328 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" event={"ID":"555ef1b2-e232-4169-818b-7fd2d8d18f4d","Type":"ContainerDied","Data":"48d4daa22e1696a69007e5eaed5f17a40d2b13987305878fdda6129891c3e248"} Jul 07 10:40:47.880891 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.880403 2581 scope.go:117] "RemoveContainer" containerID="eb31c162f9d9d6746bb63456c06012835a2f5f35c989db557fe0edcdcca8dbeb" Jul 07 10:40:47.880891 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.880345 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-f8tq7" Jul 07 10:40:47.956640 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.956605 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:40:47.965985 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:47.965958 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-f8tq7"] Jul 07 10:40:48.634477 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.634442 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:40:48.763691 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.763657 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-lbhrl"] Jul 07 10:40:48.764191 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764174 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17c1577-7353-44ba-ab30-d51405d36ac4" containerName="authorino" Jul 07 10:40:48.764297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764195 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17c1577-7353-44ba-ab30-d51405d36ac4" containerName="authorino" Jul 07 10:40:48.764297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764205 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" containerName="manager" Jul 07 10:40:48.764297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764211 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" containerName="manager" Jul 07 10:40:48.764297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764279 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17c1577-7353-44ba-ab30-d51405d36ac4" containerName="authorino" Jul 07 10:40:48.764297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.764291 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" containerName="manager" Jul 07 10:40:48.768356 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.768341 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:48.771275 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.771254 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 10:40:48.772321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.772302 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ctqxp\"" Jul 07 10:40:48.772424 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.772348 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 10:40:48.779952 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.779932 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-lbhrl"] Jul 07 10:40:48.883742 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.883707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:48.883742 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.883741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74gbl\" (UniqueName: \"kubernetes.io/projected/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-kube-api-access-74gbl\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:48.985314 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.985225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:48.985314 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.985261 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74gbl\" (UniqueName: \"kubernetes.io/projected/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-kube-api-access-74gbl\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:48.987824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:48.987779 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-webhook-cert\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:49.003228 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.003204 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74gbl\" (UniqueName: \"kubernetes.io/projected/182ce1ca-b7ca-4886-bacb-7d69bbd5d37c-kube-api-access-74gbl\") pod \"maas-controller-5bcd9f7d5c-lbhrl\" (UID: \"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c\") " pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:49.078086 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.078056 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:49.080183 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.080158 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555ef1b2-e232-4169-818b-7fd2d8d18f4d" path="/var/lib/kubelet/pods/555ef1b2-e232-4169-818b-7fd2d8d18f4d/volumes" Jul 07 10:40:49.247908 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.247882 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bcd9f7d5c-lbhrl"] Jul 07 10:40:49.250171 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:40:49.250136 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182ce1ca_b7ca_4886_bacb_7d69bbd5d37c.slice/crio-1bfb1082ce4055fdf9056938b7b5c610369d7d8df5f62fbc63e2973d7beeb51f WatchSource:0}: Error finding container 1bfb1082ce4055fdf9056938b7b5c610369d7d8df5f62fbc63e2973d7beeb51f: Status 404 returned error can't find the container with id 1bfb1082ce4055fdf9056938b7b5c610369d7d8df5f62fbc63e2973d7beeb51f Jul 07 10:40:49.893244 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.893205 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" event={"ID":"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c","Type":"ContainerStarted","Data":"ac57eee2e70f9efa019543fd202fe4efd96bab6a060b8bd4e3f270f288f6584f"} Jul 07 10:40:49.893682 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.893256 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" event={"ID":"182ce1ca-b7ca-4886-bacb-7d69bbd5d37c","Type":"ContainerStarted","Data":"1bfb1082ce4055fdf9056938b7b5c610369d7d8df5f62fbc63e2973d7beeb51f"} Jul 07 10:40:49.893682 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.893287 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:40:49.921701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:40:49.921650 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" podStartSLOduration=1.410586014 podStartE2EDuration="1.921634245s" totalImagesPullingTime="511.048231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:40:49.918669916 +0000 UTC m=+817.379204680" watchObservedRunningTime="2026-07-07 10:40:49.921634245 +0000 UTC m=+817.382169031" Jul 07 10:41:00.904489 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:00.904457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bcd9f7d5c-lbhrl" Jul 07 10:41:29.312344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:29.312308 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:29.410231 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:29.410190 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:29.552981 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:29.552951 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:35.463462 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:35.463412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:35.535335 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:35.535308 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:35.636034 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:35.635987 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:35.729468 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:35.729376 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:35.835211 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:35.835175 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:41.818218 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:41.818178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:41:41.911233 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:41:41.911199 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:42:36.063024 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:42:36.062982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:42:36.192204 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:42:36.192159 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:42:50.237067 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:42:50.237025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:42:51.666168 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:42:51.666130 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:43:05.913310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:43:05.913269 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:43:24.436539 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:43:24.436495 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:44:48.383815 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:44:48.383753 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:44:48.484876 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:44:48.484838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:30.636821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:30.636752 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:30.794656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:30.794605 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:31.166071 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:31.166032 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:31.286374 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:31.286336 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:41.702370 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:41.702332 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:45:41.817049 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:41.817010 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:41.917476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:41.917436 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:45:42.017197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:45:42.017109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:46:06.723986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:06.723944 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7448f57fc7-nqkww" podUID="122cc752-8177-4f01-a8e4-1e35ba39034d" containerName="console" containerID="cri-o://fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6" gracePeriod=15 Jul 07 10:46:06.969247 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:06.969214 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:46:07.017458 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017376 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017458 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017417 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017479 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017505 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77sw\" (UniqueName: \"kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017539 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017593 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017636 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert\") pod \"122cc752-8177-4f01-a8e4-1e35ba39034d\" (UID: \"122cc752-8177-4f01-a8e4-1e35ba39034d\") " Jul 07 10:46:07.017964 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017791 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca" (OuterVolumeSpecName: "service-ca") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:46:07.017964 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.017893 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:46:07.018078 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.018068 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-service-ca\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.018130 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.018086 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-trusted-ca-bundle\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.018257 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.018123 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:46:07.018369 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.018291 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config" (OuterVolumeSpecName: "console-config") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:46:07.020236 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.020209 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw" (OuterVolumeSpecName: "kube-api-access-s77sw") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "kube-api-access-s77sw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:46:07.020443 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.020420 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:46:07.020547 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.020458 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "122cc752-8177-4f01-a8e4-1e35ba39034d" (UID: "122cc752-8177-4f01-a8e4-1e35ba39034d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 10:46:07.118809 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.118774 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-oauth-config\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.118991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.118833 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77sw\" (UniqueName: \"kubernetes.io/projected/122cc752-8177-4f01-a8e4-1e35ba39034d-kube-api-access-s77sw\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.118991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.118849 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-console-config\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.118991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.118859 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/122cc752-8177-4f01-a8e4-1e35ba39034d-console-serving-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.118991 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.118866 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/122cc752-8177-4f01-a8e4-1e35ba39034d-oauth-serving-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:46:07.545333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.545298 2581 generic.go:356] "Generic (PLEG): container finished" podID="122cc752-8177-4f01-a8e4-1e35ba39034d" containerID="fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6" exitCode=0 Jul 07 10:46:07.545493 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.545360 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7448f57fc7-nqkww" Jul 07 10:46:07.545493 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.545387 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7448f57fc7-nqkww" event={"ID":"122cc752-8177-4f01-a8e4-1e35ba39034d","Type":"ContainerDied","Data":"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6"} Jul 07 10:46:07.545493 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.545470 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7448f57fc7-nqkww" event={"ID":"122cc752-8177-4f01-a8e4-1e35ba39034d","Type":"ContainerDied","Data":"177c668ae43b2d9b299432912683294a78f6b05aab40c56e28ec42ae95c89dfb"} Jul 07 10:46:07.545653 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.545496 2581 scope.go:117] "RemoveContainer" containerID="fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6" Jul 07 10:46:07.554414 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.554392 2581 scope.go:117] "RemoveContainer" containerID="fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6" Jul 07 10:46:07.554676 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:46:07.554656 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6\": container with ID starting with fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6 not found: ID does not exist" containerID="fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6" Jul 07 10:46:07.554728 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.554684 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6"} err="failed to get container status \"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6\": rpc error: code = NotFound desc = could not find container \"fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6\": container with ID starting with fa22faeb9035b0f0cedb79f875ecef5f12508c3759f387aae97304ca8a8c99b6 not found: ID does not exist" Jul 07 10:46:07.564932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.564908 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:46:07.570266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:07.570244 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7448f57fc7-nqkww"] Jul 07 10:46:09.079870 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:09.079838 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="122cc752-8177-4f01-a8e4-1e35ba39034d" path="/var/lib/kubelet/pods/122cc752-8177-4f01-a8e4-1e35ba39034d/volumes" Jul 07 10:46:38.065391 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:46:38.065281 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:50:01.851021 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:50:01.850988 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:50:03.887843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:50:03.887783 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:50:29.606215 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:50:29.606146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:14.059415 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:14.059372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:19.439045 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:19.439007 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:28.446704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:28.446667 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:32.926410 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:32.926351 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:36.183348 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:36.183248 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:36.277520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:36.277477 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:36.693459 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:36.693421 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:43.123009 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.122963 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:43.229321 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.229284 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:43.326729 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.326703 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:43.327177 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.327164 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="122cc752-8177-4f01-a8e4-1e35ba39034d" containerName="console" Jul 07 10:52:43.327222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.327180 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="122cc752-8177-4f01-a8e4-1e35ba39034d" containerName="console" Jul 07 10:52:43.327278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.327266 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="122cc752-8177-4f01-a8e4-1e35ba39034d" containerName="console" Jul 07 10:52:43.330437 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.330416 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.333923 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.333900 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-07962f32-openshift-default-dockercfg-lf4sf\"" Jul 07 10:52:43.359738 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.359706 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:43.361425 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.361396 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:43.393447 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393360 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393447 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bg72\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393447 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393433 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393482 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393574 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393722 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393691 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393974 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.393974 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.393774 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494435 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494449 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bg72\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494630 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.494947 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494716 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.495090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.494982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.495090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.495037 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.495090 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.495069 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.495621 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.495592 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.495750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.495726 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.497075 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.497052 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.497237 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.497222 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.508160 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.508134 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bg72\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.517686 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.517658 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:43.521319 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.521296 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token\") pod \"e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.641610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.641581 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:43.794260 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.794236 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:43.795902 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:52:43.795873 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bfa488_d5bd_4fac_80bc_5b740c8d4710.slice/crio-9893d226ee8dd64b12bb45c1db0cc477560019571153044a309309891fb5f033 WatchSource:0}: Error finding container 9893d226ee8dd64b12bb45c1db0cc477560019571153044a309309891fb5f033: Status 404 returned error can't find the container with id 9893d226ee8dd64b12bb45c1db0cc477560019571153044a309309891fb5f033 Jul 07 10:52:43.797901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.797868 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:43.797989 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.797934 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:43.797989 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:43.797963 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:44.559764 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.559723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:44.573566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.573530 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" event={"ID":"73bfa488-d5bd-4fac-80bc-5b740c8d4710","Type":"ContainerStarted","Data":"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29"} Jul 07 10:52:44.573723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.573585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" event={"ID":"73bfa488-d5bd-4fac-80bc-5b740c8d4710","Type":"ContainerStarted","Data":"9893d226ee8dd64b12bb45c1db0cc477560019571153044a309309891fb5f033"} Jul 07 10:52:44.609643 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.609549 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" podStartSLOduration=1.609529517 podStartE2EDuration="1.609529517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:52:44.605134224 +0000 UTC m=+1532.065668994" watchObservedRunningTime="2026-07-07 10:52:44.609529517 +0000 UTC m=+1532.070064275" Jul 07 10:52:44.641901 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.641865 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:44.647440 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.647415 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:44.669015 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.668982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:44.783298 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:44.783255 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:45.014050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:45.014009 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:45.222181 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:45.222137 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:45.318735 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:45.318642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:45.579109 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:45.579028 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:45.579997 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:45.579979 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:47.123495 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:47.123457 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:49.718611 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:49.718574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:49.849570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:49.849534 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:49.849746 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:49.849719 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" podUID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" containerName="istio-proxy" containerID="cri-o://7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29" gracePeriod=30 Jul 07 10:52:49.890056 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:49.890019 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:50.018588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:50.018494 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:50.194183 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:50.194146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:51.591156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:51.591125 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:55.120604 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.120579 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:55.209096 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209060 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209270 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209112 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209270 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209217 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209385 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209283 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209385 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209316 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bg72\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209385 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209344 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209545 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209391 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:52:55.209545 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209545 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209468 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209545 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209531 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data\") pod \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\" (UID: \"73bfa488-d5bd-4fac-80bc-5b740c8d4710\") " Jul 07 10:52:55.209823 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209537 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:52:55.209823 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209587 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:52:55.209823 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209663 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:52:55.209986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209866 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.209986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209867 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data" (OuterVolumeSpecName: "istio-data") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:52:55.209986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209890 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.209986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209902 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.209986 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.209910 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.211744 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.211700 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token" (OuterVolumeSpecName: "istio-token") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:52:55.211875 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.211834 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:52:55.212098 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.212072 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 10:52:55.212197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.212100 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72" (OuterVolumeSpecName: "kube-api-access-4bg72") pod "73bfa488-d5bd-4fac-80bc-5b740c8d4710" (UID: "73bfa488-d5bd-4fac-80bc-5b740c8d4710"). InnerVolumeSpecName "kube-api-access-4bg72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:52:55.310856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.310790 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.310856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.310849 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.310856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.310859 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bg72\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-kube-api-access-4bg72\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.310856 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.310869 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.311122 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.310877 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73bfa488-d5bd-4fac-80bc-5b740c8d4710-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:52:55.629842 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.629810 2581 generic.go:356] "Generic (PLEG): container finished" podID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" containerID="7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29" exitCode=0 Jul 07 10:52:55.630021 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.629863 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" Jul 07 10:52:55.630021 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.629906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" event={"ID":"73bfa488-d5bd-4fac-80bc-5b740c8d4710","Type":"ContainerDied","Data":"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29"} Jul 07 10:52:55.630021 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.629959 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk" event={"ID":"73bfa488-d5bd-4fac-80bc-5b740c8d4710","Type":"ContainerDied","Data":"9893d226ee8dd64b12bb45c1db0cc477560019571153044a309309891fb5f033"} Jul 07 10:52:55.630021 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.629983 2581 scope.go:117] "RemoveContainer" containerID="7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29" Jul 07 10:52:55.640845 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.640790 2581 scope.go:117] "RemoveContainer" containerID="7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29" Jul 07 10:52:55.643197 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:52:55.643160 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29\": container with ID starting with 7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29 not found: ID does not exist" containerID="7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29" Jul 07 10:52:55.643336 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.643207 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29"} err="failed to get container status \"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29\": rpc error: code = NotFound desc = could not find container \"7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29\": container with ID starting with 7ac082e9bc71c63b6da1e992194ccfb6b24b7f16e618b34ef7e946be1fb61b29 not found: ID does not exist" Jul 07 10:52:55.662226 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.662194 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:55.667469 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:55.667445 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-07962f32-openshift-default-5f889ddf7c-sg7fk"] Jul 07 10:52:56.108222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.108134 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:56.267728 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.267692 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:56.473752 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.473709 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:52:56.474238 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.474219 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" containerName="istio-proxy" Jul 07 10:52:56.474337 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.474241 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" containerName="istio-proxy" Jul 07 10:52:56.474392 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.474341 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" containerName="istio-proxy" Jul 07 10:52:56.478702 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.478678 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.483787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.483763 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e1f095cc-openshift-default-dockercfg-f8r9r\"" Jul 07 10:52:56.524555 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.524516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktwr9\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.524852 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.524685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:56.524975 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.524837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525109 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525248 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525391 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525391 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.525570 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.525493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.526950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.526926 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:52:56.614485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.614447 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:56.626540 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.626704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626546 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.626704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626687 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.626848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626732 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.626848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.626960 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626916 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627023 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626958 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627023 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.626995 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627023 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627046 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktwr9\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627121 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.627576 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.627555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.629224 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.629201 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.629476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.629459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.636409 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.636384 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.636640 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.636620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktwr9\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9\") pod \"e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.788910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.788789 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:56.938058 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.938032 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:52:56.939737 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:52:56.939706 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924ca769_4d78_4020_b017_9ab91438a83d.slice/crio-4936881204ddac88fb9cde1eeb1ded377c90712af32d74eeb7ad2e27c259134f WatchSource:0}: Error finding container 4936881204ddac88fb9cde1eeb1ded377c90712af32d74eeb7ad2e27c259134f: Status 404 returned error can't find the container with id 4936881204ddac88fb9cde1eeb1ded377c90712af32d74eeb7ad2e27c259134f Jul 07 10:52:56.941945 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.941911 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:56.942016 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.941975 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:56.942016 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:56.942001 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:52:57.082232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.082140 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bfa488-d5bd-4fac-80bc-5b740c8d4710" path="/var/lib/kubelet/pods/73bfa488-d5bd-4fac-80bc-5b740c8d4710/volumes" Jul 07 10:52:57.576518 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.576476 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:57.644773 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.644736 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerStarted","Data":"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46"} Jul 07 10:52:57.644962 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.644790 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerStarted","Data":"4936881204ddac88fb9cde1eeb1ded377c90712af32d74eeb7ad2e27c259134f"} Jul 07 10:52:57.669769 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.669718 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" podStartSLOduration=1.669701747 podStartE2EDuration="1.669701747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:52:57.667610692 +0000 UTC m=+1545.128145461" watchObservedRunningTime="2026-07-07 10:52:57.669701747 +0000 UTC m=+1545.130236556" Jul 07 10:52:57.674330 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.674299 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:57.789295 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.789264 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:57.823850 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:57.823812 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:58.063009 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:58.062975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:58.515032 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:58.514999 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:58.609549 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:58.609512 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:58.794607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:58.794529 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:59.602017 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:59.601981 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:52:59.657704 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:59.657657 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:52:59.658890 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:52:59.658864 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:53:02.674207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:02.674174 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:03.095399 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:03.095292 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:08.708538 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.708504 2581 generic.go:356] "Generic (PLEG): container finished" podID="924ca769-4d78-4020-b017-9ab91438a83d" containerID="50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46" exitCode=0 Jul 07 10:53:08.709030 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.708578 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerDied","Data":"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46"} Jul 07 10:53:08.709074 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.709031 2581 scope.go:117] "RemoveContainer" containerID="50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46" Jul 07 10:53:08.710098 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.710065 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:08.710208 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.710132 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:08.710208 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.710164 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:08.801546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.801512 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:53:08.832377 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.832341 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:08.933288 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:08.933249 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:09.716327 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:09.716288 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerStarted","Data":"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35"} Jul 07 10:53:09.716765 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:09.716369 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" containerID="cri-o://4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35" gracePeriod=30 Jul 07 10:53:11.104576 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:11.104530 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:14.658152 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:14.658120 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:53:14.980563 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:14.980542 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:53:15.106791 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106761 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktwr9\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.106965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106859 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.106965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106885 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.106965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106909 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.106965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106925 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.106965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.106957 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.107209 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107013 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.107209 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107056 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.107209 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107084 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs\") pod \"924ca769-4d78-4020-b017-9ab91438a83d\" (UID: \"924ca769-4d78-4020-b017-9ab91438a83d\") " Jul 07 10:53:15.107360 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107252 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:15.107360 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107328 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:53:15.107521 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107475 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:15.107594 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107502 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.107594 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107537 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:15.107594 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107550 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/924ca769-4d78-4020-b017-9ab91438a83d-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.108081 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.107997 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data" (OuterVolumeSpecName: "istio-data") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:15.109144 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.109118 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9" (OuterVolumeSpecName: "kube-api-access-ktwr9") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "kube-api-access-ktwr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:53:15.109282 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.109203 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token" (OuterVolumeSpecName: "istio-token") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:53:15.109344 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.109294 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 10:53:15.109487 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.109471 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "924ca769-4d78-4020-b017-9ab91438a83d" (UID: "924ca769-4d78-4020-b017-9ab91438a83d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208367 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208400 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208412 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208423 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktwr9\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-kube-api-access-ktwr9\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208435 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/924ca769-4d78-4020-b017-9ab91438a83d-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208446 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/924ca769-4d78-4020-b017-9ab91438a83d-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.208471 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.208457 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/924ca769-4d78-4020-b017-9ab91438a83d-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:15.749579 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.749541 2581 generic.go:356] "Generic (PLEG): container finished" podID="924ca769-4d78-4020-b017-9ab91438a83d" containerID="4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35" exitCode=0 Jul 07 10:53:15.750100 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.749596 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" Jul 07 10:53:15.750100 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.749631 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerDied","Data":"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35"} Jul 07 10:53:15.750100 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.749689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh" event={"ID":"924ca769-4d78-4020-b017-9ab91438a83d","Type":"ContainerDied","Data":"4936881204ddac88fb9cde1eeb1ded377c90712af32d74eeb7ad2e27c259134f"} Jul 07 10:53:15.750100 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.749713 2581 scope.go:117] "RemoveContainer" containerID="4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35" Jul 07 10:53:15.759263 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.759245 2581 scope.go:117] "RemoveContainer" containerID="50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46" Jul 07 10:53:15.767281 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.767265 2581 scope.go:117] "RemoveContainer" containerID="4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35" Jul 07 10:53:15.767534 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:53:15.767515 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35\": container with ID starting with 4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35 not found: ID does not exist" containerID="4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35" Jul 07 10:53:15.767603 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.767548 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35"} err="failed to get container status \"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35\": rpc error: code = NotFound desc = could not find container \"4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35\": container with ID starting with 4231255cea251deee99ec3a4a58d6ecfe290332b09d47d823ba142c08ef9ac35 not found: ID does not exist" Jul 07 10:53:15.767603 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.767573 2581 scope.go:117] "RemoveContainer" containerID="50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46" Jul 07 10:53:15.767858 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:53:15.767840 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46\": container with ID starting with 50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46 not found: ID does not exist" containerID="50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46" Jul 07 10:53:15.767908 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.767864 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46"} err="failed to get container status \"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46\": rpc error: code = NotFound desc = could not find container \"50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46\": container with ID starting with 50399c590dd405c3657c51484716928cff6dccbe7d4e02bd12a6c0df16a93d46 not found: ID does not exist" Jul 07 10:53:15.776132 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.776108 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:53:15.781778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.781756 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e1f095cc-openshift-default-5cb7bfbb4c-q2twh"] Jul 07 10:53:15.944583 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:15.944549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:16.105735 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.105647 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:16.248336 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248307 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:16.248742 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248729 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.248787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248745 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.248787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248760 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.248787 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248765 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.248910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248857 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.248910 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.248866 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="924ca769-4d78-4020-b017-9ab91438a83d" containerName="istio-proxy" Jul 07 10:53:16.253716 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.253697 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.257739 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.257711 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-593c3056-openshift-default-dockercfg-dkdwq\"" Jul 07 10:53:16.278985 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.278962 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:16.353292 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.353262 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:16.421104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421363 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421363 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421188 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421363 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421363 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421314 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421363 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rxmr\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.421624 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.421397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.483917 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.483879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:16.522601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.522838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.522838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.522838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522677 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.522838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522704 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.522838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rxmr\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522876 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523050 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.522909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523231 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.523191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523305 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.523233 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523394 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.523377 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523455 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.523400 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.523723 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.523701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.525195 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.525178 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.525340 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.525324 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.544300 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.544272 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rxmr\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.545146 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.545123 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token\") pod \"e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.564241 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.564211 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:16.732042 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.732017 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:16.734381 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:53:16.734349 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c09a57_fc5f_462a_a0db_114d3f475281.slice/crio-a9efa7f396e2ab867375d6e467fb71c41ac630bf61443d4bbb0ed7c9ebdeedea WatchSource:0}: Error finding container a9efa7f396e2ab867375d6e467fb71c41ac630bf61443d4bbb0ed7c9ebdeedea: Status 404 returned error can't find the container with id a9efa7f396e2ab867375d6e467fb71c41ac630bf61443d4bbb0ed7c9ebdeedea Jul 07 10:53:16.736562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.736528 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:16.736647 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.736598 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:16.736647 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.736631 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:53:16.757134 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:16.757102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" event={"ID":"69c09a57-fc5f-462a-a0db-114d3f475281","Type":"ContainerStarted","Data":"a9efa7f396e2ab867375d6e467fb71c41ac630bf61443d4bbb0ed7c9ebdeedea"} Jul 07 10:53:17.080023 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.079928 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="924ca769-4d78-4020-b017-9ab91438a83d" path="/var/lib/kubelet/pods/924ca769-4d78-4020-b017-9ab91438a83d/volumes" Jul 07 10:53:17.478969 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.478926 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:17.647701 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.647667 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:17.764134 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.764044 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" event={"ID":"69c09a57-fc5f-462a-a0db-114d3f475281","Type":"ContainerStarted","Data":"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b"} Jul 07 10:53:17.828003 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.827884 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" podStartSLOduration=1.82786264 podStartE2EDuration="1.82786264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:53:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:53:17.82728725 +0000 UTC m=+1565.287821995" watchObservedRunningTime="2026-07-07 10:53:17.82786264 +0000 UTC m=+1565.288397385" Jul 07 10:53:17.893072 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:17.893034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:18.128639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.128604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:18.564511 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.564425 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:18.569523 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.569495 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:18.769566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.769525 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:18.770501 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.770480 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:18.951296 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:18.951262 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:19.073755 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:19.073717 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:19.266000 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:19.265965 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:21.926710 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:21.926661 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:22.138197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:22.138148 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:22.138412 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:22.138373 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" podUID="69c09a57-fc5f-462a-a0db-114d3f475281" containerName="istio-proxy" containerID="cri-o://171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b" gracePeriod=30 Jul 07 10:53:22.154735 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:22.154690 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:22.295779 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:22.295677 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:22.589329 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:22.589234 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:23.604950 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:23.604920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:29.097843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.097771 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:29.145769 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145744 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145787 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145840 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145859 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145882 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.145938 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145920 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.146266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145948 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rxmr\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.146266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.145980 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert\") pod \"69c09a57-fc5f-462a-a0db-114d3f475281\" (UID: \"69c09a57-fc5f-462a-a0db-114d3f475281\") " Jul 07 10:53:29.146266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146037 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:29.146266 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146170 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:29.146476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146383 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.146476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146404 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.146865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146611 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:29.146988 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146811 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:53:29.146988 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.146894 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data" (OuterVolumeSpecName: "istio-data") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:29.148314 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.148285 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 10:53:29.148747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.148728 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:53:29.148903 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.148878 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token" (OuterVolumeSpecName: "istio-token") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:53:29.148966 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.148935 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr" (OuterVolumeSpecName: "kube-api-access-4rxmr") pod "69c09a57-fc5f-462a-a0db-114d3f475281" (UID: "69c09a57-fc5f-462a-a0db-114d3f475281"). InnerVolumeSpecName "kube-api-access-4rxmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:53:29.246952 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246916 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.246952 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246943 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69c09a57-fc5f-462a-a0db-114d3f475281-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.246952 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246959 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.247205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246971 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rxmr\" (UniqueName: \"kubernetes.io/projected/69c09a57-fc5f-462a-a0db-114d3f475281-kube-api-access-4rxmr\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.247205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246981 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69c09a57-fc5f-462a-a0db-114d3f475281-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.247205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246989 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.247205 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.246997 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69c09a57-fc5f-462a-a0db-114d3f475281-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:53:29.247916 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.247886 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:29.366522 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.366426 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:29.811750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.811710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:29.842520 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.842483 2581 generic.go:356] "Generic (PLEG): container finished" podID="69c09a57-fc5f-462a-a0db-114d3f475281" containerID="171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b" exitCode=0 Jul 07 10:53:29.842711 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.842545 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" Jul 07 10:53:29.842711 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.842567 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" event={"ID":"69c09a57-fc5f-462a-a0db-114d3f475281","Type":"ContainerDied","Data":"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b"} Jul 07 10:53:29.842711 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.842629 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5" event={"ID":"69c09a57-fc5f-462a-a0db-114d3f475281","Type":"ContainerDied","Data":"a9efa7f396e2ab867375d6e467fb71c41ac630bf61443d4bbb0ed7c9ebdeedea"} Jul 07 10:53:29.842711 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.842652 2581 scope.go:117] "RemoveContainer" containerID="171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b" Jul 07 10:53:29.853012 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.852989 2581 scope.go:117] "RemoveContainer" containerID="171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b" Jul 07 10:53:29.853393 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:53:29.853362 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b\": container with ID starting with 171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b not found: ID does not exist" containerID="171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b" Jul 07 10:53:29.853485 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.853405 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b"} err="failed to get container status \"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b\": rpc error: code = NotFound desc = could not find container \"171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b\": container with ID starting with 171fe6aa57c9f3ee33f931a98f4d00c9e97d7b7d00bfa22f4bf1c182abb64e9b not found: ID does not exist" Jul 07 10:53:29.878010 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.877976 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:29.887317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.887284 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-593c3056-openshift-default-57ccffc5c5-g7qc5"] Jul 07 10:53:29.931461 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:29.931431 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:30.087278 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:30.087184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:31.080061 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:31.080025 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69c09a57-fc5f-462a-a0db-114d3f475281" path="/var/lib/kubelet/pods/69c09a57-fc5f-462a-a0db-114d3f475281/volumes" Jul 07 10:53:31.633229 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:31.633191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:37.506747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:37.506714 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:37.607080 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:37.607032 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:37.705406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:37.705373 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:37.805350 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:37.805254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:53:38.263515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:53:38.263484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:04.312438 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:04.312355 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:10.005965 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:10.005928 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:10.067848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:10.067783 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:10.145596 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:10.145548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:10.247658 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:10.247623 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:14.165105 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:14.165063 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:26.170643 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:26.170606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:32.754405 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:32.754342 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:32.922158 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:32.922109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:33.003387 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:33.003353 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:43.494846 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:43.494787 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:44.320775 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:44.320743 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:50.150396 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:50.150354 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:50.306243 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:50.306195 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:50.387866 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:50.387836 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:50.487948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:50.487849 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:51.275311 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:51.275279 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:51.373103 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:51.373064 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:51.473385 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:51.473343 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:53.216852 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:53.216793 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:59.222853 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:59.222758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:59.324297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:59.324261 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:54:59.945529 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:54:59.945484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:05.753154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:05.753098 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:06.313688 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.313653 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:06.314166 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.314148 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69c09a57-fc5f-462a-a0db-114d3f475281" containerName="istio-proxy" Jul 07 10:55:06.314264 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.314169 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69c09a57-fc5f-462a-a0db-114d3f475281" containerName="istio-proxy" Jul 07 10:55:06.314264 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.314254 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69c09a57-fc5f-462a-a0db-114d3f475281" containerName="istio-proxy" Jul 07 10:55:06.317454 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.317431 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.320364 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.320342 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b69a63f5-openshift-default-dockercfg-qh52h\"" Jul 07 10:55:06.337740 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.337709 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:06.365493 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365455 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57fk9\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.365948 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.365775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466335 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466365 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466424 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57fk9\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466970 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466742 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466970 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466859 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.466970 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.466928 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.467143 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.467016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.467231 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.467208 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.468979 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.468957 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.469117 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.469101 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.481779 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.481752 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.491641 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.491608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57fk9\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9\") pod \"e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.628421 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.628387 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:06.780915 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.780887 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:06.782778 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:55:06.782750 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949d6035_1193_4ebc_b4b7_57c62ee8f78f.slice/crio-631395765a92742fec9c85ddc1490391771746137673a620551fa79d4506eacf WatchSource:0}: Error finding container 631395765a92742fec9c85ddc1490391771746137673a620551fa79d4506eacf: Status 404 returned error can't find the container with id 631395765a92742fec9c85ddc1490391771746137673a620551fa79d4506eacf Jul 07 10:55:06.785019 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.784985 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:06.785133 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.785054 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:06.785133 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:06.785089 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:07.019204 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.019061 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:07.328317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.328216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerStarted","Data":"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1"} Jul 07 10:55:07.328317 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.328276 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerStarted","Data":"631395765a92742fec9c85ddc1490391771746137673a620551fa79d4506eacf"} Jul 07 10:55:07.351766 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.351714 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podStartSLOduration=1.351698147 podStartE2EDuration="1.351698147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:55:07.349332273 +0000 UTC m=+1674.809867033" watchObservedRunningTime="2026-07-07 10:55:07.351698147 +0000 UTC m=+1674.812232892" Jul 07 10:55:07.629018 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.628977 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:07.634211 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:07.634182 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:08.332745 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:08.332715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:08.333677 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:08.333659 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:09.292543 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:09.292501 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:09.387607 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:09.387568 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:09.518550 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:09.518512 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:09.845738 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:09.845699 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:10.015768 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:10.015733 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:10.114970 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:10.114873 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:13.846396 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:13.846350 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:14.525157 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:14.525090 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:14.623565 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:14.623503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:14.982824 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:14.982772 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:20.941539 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:20.941500 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:21.004357 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:21.004312 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:21.234441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:21.234324 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:21.704559 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:21.704515 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:21.775982 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:21.775947 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:22.404825 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.404767 2581 generic.go:356] "Generic (PLEG): container finished" podID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerID="c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1" exitCode=0 Jul 07 10:55:22.405269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.404848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerDied","Data":"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1"} Jul 07 10:55:22.405269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.405248 2581 scope.go:117] "RemoveContainer" containerID="c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1" Jul 07 10:55:22.406236 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.406207 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:22.406328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.406273 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:22.406328 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:22.406308 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:23.333678 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:23.333648 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:23.411501 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:23.411466 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerStarted","Data":"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba"} Jul 07 10:55:23.629033 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:23.628985 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:23.630238 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:23.630205 2581 patch_prober.go:28] interesting pod/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 07 10:55:23.630399 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:23.630290 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 07 10:55:24.629499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:24.629463 2581 patch_prober.go:28] interesting pod/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 07 10:55:24.630142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:24.630110 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 07 10:55:25.629190 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:25.629143 2581 patch_prober.go:28] interesting pod/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 07 10:55:25.629415 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:25.629214 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 07 10:55:26.629655 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:26.629612 2581 patch_prober.go:28] interesting pod/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 07 10:55:26.630314 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:26.630280 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 07 10:55:27.629010 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:27.628972 2581 patch_prober.go:28] interesting pod/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 07 10:55:27.629186 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:27.629047 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 07 10:55:27.869789 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:27.869750 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:27.968086 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:27.967985 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:28.106207 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.106173 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:28.106381 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.106358 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" containerID="cri-o://7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba" gracePeriod=30 Jul 07 10:55:28.121847 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.121772 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:28.362708 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.362606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:28.492747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.492706 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:28.595939 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.595906 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:28.721461 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:28.721422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:33.360603 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.360579 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:33.434518 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434482 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434525 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434559 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434617 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434639 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434673 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.434717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434703 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.435060 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434774 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57fk9\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.435060 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.434828 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs\") pod \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\" (UID: \"949d6035-1193-4ebc-b4b7-57c62ee8f78f\") " Jul 07 10:55:33.435667 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.435439 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:55:33.435667 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.435637 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:55:33.438333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.438295 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:55:33.438333 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.438324 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token" (OuterVolumeSpecName: "istio-token") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:55:33.438503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.438401 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:55:33.438503 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.438477 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:55:33.438610 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.438580 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data" (OuterVolumeSpecName: "istio-data") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:55:33.440148 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.440109 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 10:55:33.440631 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.440607 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9" (OuterVolumeSpecName: "kube-api-access-57fk9") pod "949d6035-1193-4ebc-b4b7-57c62ee8f78f" (UID: "949d6035-1193-4ebc-b4b7-57c62ee8f78f"). InnerVolumeSpecName "kube-api-access-57fk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:55:33.472366 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.472334 2581 generic.go:356] "Generic (PLEG): container finished" podID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerID="7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba" exitCode=0 Jul 07 10:55:33.472510 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.472392 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" Jul 07 10:55:33.472510 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.472424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerDied","Data":"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba"} Jul 07 10:55:33.472510 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.472486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp" event={"ID":"949d6035-1193-4ebc-b4b7-57c62ee8f78f","Type":"ContainerDied","Data":"631395765a92742fec9c85ddc1490391771746137673a620551fa79d4506eacf"} Jul 07 10:55:33.472682 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.472514 2581 scope.go:117] "RemoveContainer" containerID="7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba" Jul 07 10:55:33.481456 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.481438 2581 scope.go:117] "RemoveContainer" containerID="c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1" Jul 07 10:55:33.489508 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.489492 2581 scope.go:117] "RemoveContainer" containerID="7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba" Jul 07 10:55:33.489782 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:55:33.489763 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba\": container with ID starting with 7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba not found: ID does not exist" containerID="7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba" Jul 07 10:55:33.489843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.489812 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba"} err="failed to get container status \"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba\": rpc error: code = NotFound desc = could not find container \"7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba\": container with ID starting with 7b87195d387f7e1b583d44520dde353db56c4b7ecc909ac950ee32967347d1ba not found: ID does not exist" Jul 07 10:55:33.489843 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.489834 2581 scope.go:117] "RemoveContainer" containerID="c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1" Jul 07 10:55:33.490084 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:55:33.490069 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1\": container with ID starting with c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1 not found: ID does not exist" containerID="c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1" Jul 07 10:55:33.490125 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.490087 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1"} err="failed to get container status \"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1\": rpc error: code = NotFound desc = could not find container \"c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1\": container with ID starting with c9df722afa4b9c6d04c64bb28f229ae13976b7009c5a5353a04072f2879b50b1 not found: ID does not exist" Jul 07 10:55:33.504191 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.504169 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:33.510431 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.510409 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b69a63f5-openshift-default-646dbd8559-rsfjp"] Jul 07 10:55:33.536032 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.535992 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57fk9\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-kube-api-access-57fk9\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536032 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536029 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536032 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536040 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536048 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536058 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536066 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536074 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536082 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:33.536222 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:33.536089 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949d6035-1193-4ebc-b4b7-57c62ee8f78f-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:55:35.069814 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:35.069763 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:35.080192 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:35.080164 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" path="/var/lib/kubelet/pods/949d6035-1193-4ebc-b4b7-57c62ee8f78f/volumes" Jul 07 10:55:35.207681 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:35.207646 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:35.365615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:35.365520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:35.512371 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:35.512333 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:41.402915 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:41.402883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:41.509255 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:41.509216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:48.413790 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.413747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:48.683293 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683197 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:55:48.683717 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683701 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:55:48.683788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683718 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:55:48.683788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683737 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:55:48.683788 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683742 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:55:48.683932 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.683817 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:55:48.687516 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.687496 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.690524 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.690502 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-17d0cfdd-openshift-default-dockercfg-b2rxl\"" Jul 07 10:55:48.699461 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.699435 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:55:48.712696 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.712661 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:48.778391 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778340 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778450 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6mwj\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778615 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778785 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.778868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.778849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.879750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.879750 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879759 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879851 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.879975 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6mwj\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880004 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880232 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880207 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880298 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880759 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880736 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.880974 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.880789 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.881239 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.881211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.883265 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.883239 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.883401 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.883382 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.888476 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.888448 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.888650 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.888630 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6mwj\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj\") pod \"e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:48.999040 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:48.998941 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:49.072566 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.072511 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:49.185324 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.185300 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:55:49.187939 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:55:49.187911 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023a9933_ca80_4870_9cec_41f0c91d3bc6.slice/crio-8910c73a64f7ea388d152ed15dd7959422c8d305eea5330df1a0d5f004facc35 WatchSource:0}: Error finding container 8910c73a64f7ea388d152ed15dd7959422c8d305eea5330df1a0d5f004facc35: Status 404 returned error can't find the container with id 8910c73a64f7ea388d152ed15dd7959422c8d305eea5330df1a0d5f004facc35 Jul 07 10:55:49.190322 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.190283 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:49.190406 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.190368 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:49.190446 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.190402 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:55:49.555399 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.555295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerStarted","Data":"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670"} Jul 07 10:55:49.555399 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.555353 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerStarted","Data":"8910c73a64f7ea388d152ed15dd7959422c8d305eea5330df1a0d5f004facc35"} Jul 07 10:55:49.578629 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.578557 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podStartSLOduration=1.578534194 podStartE2EDuration="1.578534194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:55:49.577287324 +0000 UTC m=+1717.037822094" watchObservedRunningTime="2026-07-07 10:55:49.578534194 +0000 UTC m=+1717.039068941" Jul 07 10:55:50.000036 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:49.999997 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:51.004326 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:51.004299 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:51.568102 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:51.568073 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:51.569097 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:51.569078 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:55:51.853977 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:51.853869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:52.304153 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:52.304118 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:52.403707 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:52.403662 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:55:56.047961 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:55:56.047909 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:02.640441 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:02.640341 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:06.569865 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.569792 2581 patch_prober.go:28] interesting pod/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 10:56:06.570305 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.569888 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 10:56:06.665151 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.665114 2581 generic.go:356] "Generic (PLEG): container finished" podID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerID="6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670" exitCode=0 Jul 07 10:56:06.665326 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.665194 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerDied","Data":"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670"} Jul 07 10:56:06.665656 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.665638 2581 scope.go:117] "RemoveContainer" containerID="6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670" Jul 07 10:56:06.666695 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.666664 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:56:06.666767 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.666749 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:56:06.666840 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:06.666825 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:56:07.672358 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:07.672324 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerStarted","Data":"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d"} Jul 07 10:56:08.000104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:08.000012 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:56:08.000258 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:08.000207 2581 patch_prober.go:28] interesting pod/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 10:56:08.000306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:08.000266 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 10:56:08.999918 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:08.999878 2581 patch_prober.go:28] interesting pod/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 10:56:09.000413 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:08.999942 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 10:56:09.999786 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:09.999746 2581 patch_prober.go:28] interesting pod/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 10:56:10.000006 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:09.999834 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 10:56:10.999815 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:10.999774 2581 patch_prober.go:28] interesting pod/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 10:56:10.999997 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:10.999856 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 10:56:11.398639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:11.398570 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:11.497484 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:11.497441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:12.004286 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:12.004253 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:56:12.004536 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:12.004518 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:56:12.005297 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:12.005282 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:56:12.839211 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:12.839177 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:12.939747 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:12.939707 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:33.701924 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:33.701887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:34.285364 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:34.285323 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:39.921220 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:39.921184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:56:44.988675 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:56:44.988635 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:21.121714 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:21.121675 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:22.571561 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:22.571520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:28.190146 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:28.190109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:34.047828 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.047758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:34.152821 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.152757 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:34.362985 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.362897 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:57:34.363156 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.363133 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" containerID="cri-o://63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d" gracePeriod=30 Jul 07 10:57:34.386373 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.386346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:34.603616 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:34.603578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:35.668095 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:35.668055 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:35.766601 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:35.766569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:35.865562 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:35.865524 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:36.630421 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:36.630379 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:36.724475 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:36.724437 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:36.839628 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:36.839590 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:38.305187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:38.305149 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:39.632358 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.632289 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:57:39.725165 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725133 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725165 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725167 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725200 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6mwj\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725228 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725253 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725278 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725310 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725345 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725417 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725376 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket\") pod \"023a9933-ca80-4870-9cec-41f0c91d3bc6\" (UID: \"023a9933-ca80-4870-9cec-41f0c91d3bc6\") " Jul 07 10:57:39.725778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725553 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:57:39.725778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725566 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:57:39.725778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725596 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 10:57:39.725778 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725764 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:57:39.726037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725887 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.726037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725904 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.726037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725917 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.726037 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.725930 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/023a9933-ca80-4870-9cec-41f0c91d3bc6-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.726517 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.726377 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data" (OuterVolumeSpecName: "istio-data") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:57:39.727907 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.727879 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 10:57:39.728058 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.728038 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj" (OuterVolumeSpecName: "kube-api-access-r6mwj") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "kube-api-access-r6mwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:57:39.728116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.728079 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 10:57:39.728116 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.728100 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "023a9933-ca80-4870-9cec-41f0c91d3bc6" (UID: "023a9933-ca80-4870-9cec-41f0c91d3bc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 10:57:39.827083 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.827042 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.827083 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.827077 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6mwj\" (UniqueName: \"kubernetes.io/projected/023a9933-ca80-4870-9cec-41f0c91d3bc6-kube-api-access-r6mwj\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.827083 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.827087 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.827083 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.827096 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:39.827355 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:39.827104 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/023a9933-ca80-4870-9cec-41f0c91d3bc6-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 10:57:40.152107 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.152070 2581 generic.go:356] "Generic (PLEG): container finished" podID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerID="63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d" exitCode=0 Jul 07 10:57:40.152306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.152124 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" Jul 07 10:57:40.152306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.152133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerDied","Data":"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d"} Jul 07 10:57:40.152306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.152185 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s" event={"ID":"023a9933-ca80-4870-9cec-41f0c91d3bc6","Type":"ContainerDied","Data":"8910c73a64f7ea388d152ed15dd7959422c8d305eea5330df1a0d5f004facc35"} Jul 07 10:57:40.152306 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.152213 2581 scope.go:117] "RemoveContainer" containerID="63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d" Jul 07 10:57:40.162515 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.162492 2581 scope.go:117] "RemoveContainer" containerID="6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670" Jul 07 10:57:40.170995 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.170973 2581 scope.go:117] "RemoveContainer" containerID="63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d" Jul 07 10:57:40.171284 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:57:40.171259 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d\": container with ID starting with 63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d not found: ID does not exist" containerID="63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d" Jul 07 10:57:40.171358 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.171292 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d"} err="failed to get container status \"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d\": rpc error: code = NotFound desc = could not find container \"63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d\": container with ID starting with 63f7304e3996f6728114fa95408afab0bdd615e1fb0661a942db5893b87e039d not found: ID does not exist" Jul 07 10:57:40.171358 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.171310 2581 scope.go:117] "RemoveContainer" containerID="6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670" Jul 07 10:57:40.171522 ip-10-0-139-204 kubenswrapper[2581]: E0707 10:57:40.171503 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670\": container with ID starting with 6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670 not found: ID does not exist" containerID="6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670" Jul 07 10:57:40.171588 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.171526 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670"} err="failed to get container status \"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670\": rpc error: code = NotFound desc = could not find container \"6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670\": container with ID starting with 6ba31479d255cc942099d0cb63e01c2e1a593b72fde5512b4e1b64f035bfd670 not found: ID does not exist" Jul 07 10:57:40.175310 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.175287 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:57:40.180080 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:40.180054 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-17d0cfdd-openshift-default-784bf77999-dgf2s"] Jul 07 10:57:41.079502 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:41.079468 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" path="/var/lib/kubelet/pods/023a9933-ca80-4870-9cec-41f0c91d3bc6/volumes" Jul 07 10:57:44.274536 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:44.274503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:44.347191 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:44.347155 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:44.974103 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:44.974067 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:51.023475 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:51.023430 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:51.163001 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:51.162964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:57:51.273616 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:57:51.273516 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:01.739741 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:01.739701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:02.085182 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:02.085080 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:08.427848 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:08.427781 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:23.062548 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:23.062508 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:28.764600 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:28.764553 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:28.831389 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:28.831356 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:29.143209 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.143170 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:29.443620 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.443534 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 10:58:29.443961 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.443947 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:29.444024 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.443962 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:29.444024 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.443972 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:29.444024 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.443977 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:29.444118 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.444036 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:29.444118 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.444046 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="949d6035-1193-4ebc-b4b7-57c62ee8f78f" containerName="istio-proxy" Jul 07 10:58:29.447187 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.447167 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.450036 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.450017 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-27bc88-openshift-default-dockercfg-z8vzr\"" Jul 07 10:58:29.462229 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.462203 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 10:58:29.471313 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.471280 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:29.605617 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.605877 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605623 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.605877 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.605877 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.605877 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.606087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.606087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.605953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwcxb\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.606087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.606000 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.606087 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.606022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.706748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.706647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.706748 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.706699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.706852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwcxb\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.706907 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.706940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707059 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707069 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707154 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707141 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707367 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707367 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707333 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707446 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707366 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.707772 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.707755 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.709342 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.709321 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.709662 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.709639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.723198 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.723174 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.723316 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.723174 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwcxb\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb\") pod \"e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.757829 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.757762 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:29.918197 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.918157 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 10:58:29.919571 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:58:29.919543 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5aaa703_6006_4ab6_a04d_af2ca0540098.slice/crio-86033b1b373477824bb61c1cedd32815118305068de96283f6a37859079b416d WatchSource:0}: Error finding container 86033b1b373477824bb61c1cedd32815118305068de96283f6a37859079b416d: Status 404 returned error can't find the container with id 86033b1b373477824bb61c1cedd32815118305068de96283f6a37859079b416d Jul 07 10:58:29.921642 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.921607 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:29.921739 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.921680 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:29.921739 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:29.921711 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:30.268002 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.267907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:30.414887 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.414833 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerStarted","Data":"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729"} Jul 07 10:58:30.415095 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.414933 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerStarted","Data":"86033b1b373477824bb61c1cedd32815118305068de96283f6a37859079b416d"} Jul 07 10:58:30.439294 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.439240 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podStartSLOduration=1.4392233380000001 podStartE2EDuration="1.439223338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:58:30.437627693 +0000 UTC m=+1877.898162439" watchObservedRunningTime="2026-07-07 10:58:30.439223338 +0000 UTC m=+1877.899758083" Jul 07 10:58:30.758918 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.758877 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:30.763887 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:30.763861 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:31.420837 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:31.420787 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:31.421768 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:31.421743 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:32.371995 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:32.371946 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:32.455269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:32.455238 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:32.534436 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:32.534391 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:32.633479 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:32.633371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:32.940354 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:32.940251 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:33.041005 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:33.040968 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:36.881868 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:36.881762 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:37.227689 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.227606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:37.507039 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.506947 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 10:58:37.507511 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.507492 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="023a9933-ca80-4870-9cec-41f0c91d3bc6" containerName="istio-proxy" Jul 07 10:58:37.511104 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.511080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.513935 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.513916 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-845408-openshift-default-dockercfg-4rgcp\"" Jul 07 10:58:37.526384 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.526353 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 10:58:37.538481 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.538454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:37.581115 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581086 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgs8\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581267 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581267 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581267 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581267 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.581496 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.581445 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682499 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682692 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682830 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgs8\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682830 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682851 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682885 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.682958 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682922 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.683158 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.683158 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.682992 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.683158 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.683133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.683274 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.683234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.683664 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.683641 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.685280 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.685250 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.685370 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.685355 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.691265 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.691240 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgs8\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.691454 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.691437 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token\") pod \"e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.759302 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.759205 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:37.822521 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.822489 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:37.970826 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.970782 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 10:58:37.971988 ip-10-0-139-204 kubenswrapper[2581]: W0707 10:58:37.971953 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ee4223_35e4_4150_a9b7_44c62ad0c05f.slice/crio-138a49bafb6e16e37d3261c46cbcf72b06f78453a25838060f82da32e0f85e2d WatchSource:0}: Error finding container 138a49bafb6e16e37d3261c46cbcf72b06f78453a25838060f82da32e0f85e2d: Status 404 returned error can't find the container with id 138a49bafb6e16e37d3261c46cbcf72b06f78453a25838060f82da32e0f85e2d Jul 07 10:58:37.974092 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.974057 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:37.974180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.974127 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:37.974180 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:37.974158 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:38.461497 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:38.461458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerStarted","Data":"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35"} Jul 07 10:58:38.461683 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:38.461522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerStarted","Data":"138a49bafb6e16e37d3261c46cbcf72b06f78453a25838060f82da32e0f85e2d"} Jul 07 10:58:38.483138 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:38.483080 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podStartSLOduration=1.4830620460000001 podStartE2EDuration="1.483062046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 10:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 10:58:38.481327274 +0000 UTC m=+1885.941862057" watchObservedRunningTime="2026-07-07 10:58:38.483062046 +0000 UTC m=+1885.943596790" Jul 07 10:58:38.822966 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:38.822869 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:39.828377 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:39.828343 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:40.446952 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:40.446906 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:40.472810 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:40.472770 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:40.473626 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:40.473607 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:40.547433 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:40.547394 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:40.736906 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:40.736701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:41.046780 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:41.046670 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:41.222549 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:41.222509 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:41.332581 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:41.332484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:44.934106 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:44.934066 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:47.463012 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:47.462963 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 10:58:47.463437 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:47.463047 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 10:58:48.527780 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.526492 2581 generic.go:356] "Generic (PLEG): container finished" podID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerID="e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729" exitCode=0 Jul 07 10:58:48.527780 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.526620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerDied","Data":"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729"} Jul 07 10:58:48.527780 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.527166 2581 scope.go:117] "RemoveContainer" containerID="e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729" Jul 07 10:58:48.529826 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.529771 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:48.530112 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.530087 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:48.530276 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:48.530256 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:49.537345 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:49.537292 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerStarted","Data":"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0"} Jul 07 10:58:49.759250 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:49.759216 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:49.759639 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:49.759538 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 10:58:49.759829 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:49.759783 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 10:58:50.759000 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:50.758959 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 10:58:50.759419 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:50.759018 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 10:58:51.759141 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:51.759096 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 10:58:51.759546 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:51.759161 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 10:58:52.758707 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:52.758666 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 10:58:52.758917 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:52.758740 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 10:58:53.758921 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:53.758885 2581 patch_prober.go:28] interesting pod/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 10:58:53.759327 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:53.758946 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 10:58:54.392183 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.392138 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:54.496508 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.496478 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:54.567969 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.567938 2581 generic.go:356] "Generic (PLEG): container finished" podID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerID="f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35" exitCode=0 Jul 07 10:58:54.568140 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.567983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerDied","Data":"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35"} Jul 07 10:58:54.568424 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.568409 2581 scope.go:117] "RemoveContainer" containerID="f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35" Jul 07 10:58:54.569482 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.569450 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:54.569560 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.569542 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:54.569626 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.569584 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 10:58:54.762142 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.762065 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:54.762532 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.762493 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:54.763269 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:54.763245 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 10:58:55.472786 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:55.472750 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:55.574959 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:55.574918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerStarted","Data":"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab"} Jul 07 10:58:55.823273 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:55.823167 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:55.824072 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:55.824038 2581 patch_prober.go:28] interesting pod/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 10:58:55.824212 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:55.824103 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 10:58:56.823961 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:56.823923 2581 patch_prober.go:28] interesting pod/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 10:58:56.824838 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:56.824780 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 10:58:57.823482 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:57.823440 2581 patch_prober.go:28] interesting pod/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 10:58:57.823691 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:57.823546 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 10:58:58.604047 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:58.604007 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:58.668473 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:58.668435 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:58:58.823634 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:58.823596 2581 patch_prober.go:28] interesting pod/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 10:58:58.823963 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:58.823936 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 10:58:59.827376 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:59.827341 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:59.827743 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:59.827683 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:58:59.828284 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:58:59.828265 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 10:59:03.085119 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:59:03.085077 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:59:03.185161 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:59:03.185124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:59:05.681994 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:59:05.681961 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 10:59:46.188259 ip-10-0-139-204 kubenswrapper[2581]: I0707 10:59:46.188187 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:04.232067 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:04.231983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:05.138821 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:05.138759 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:05.537701 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:05.537612 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:05.735789 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:05.735753 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:05.884094 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:05.884065 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:05.982565 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:05.982525 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:06.167593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:06.167505 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:10.815001 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:10.814963 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:47.583442 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:47.583406 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:47.834436 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:47.834356 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 11:00:47.834624 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:47.834603 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" containerID="cri-o://8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0" gracePeriod=30 Jul 07 11:00:47.857817 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:47.857767 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:48.100951 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:48.100851 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:48.254175 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:48.254138 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:48.671548 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:48.671509 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:48.770907 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:48.770874 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:48.925547 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:48.925460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:53.116889 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.116856 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 11:00:53.189037 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.189002 2581 generic.go:356] "Generic (PLEG): container finished" podID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerID="8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0" exitCode=0 Jul 07 11:00:53.189240 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.189054 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" Jul 07 11:00:53.189240 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.189091 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerDied","Data":"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0"} Jul 07 11:00:53.189240 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.189155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk" event={"ID":"c5aaa703-6006-4ab6-a04d-af2ca0540098","Type":"ContainerDied","Data":"86033b1b373477824bb61c1cedd32815118305068de96283f6a37859079b416d"} Jul 07 11:00:53.189240 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.189186 2581 scope.go:117] "RemoveContainer" containerID="8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0" Jul 07 11:00:53.198076 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.198055 2581 scope.go:117] "RemoveContainer" containerID="e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729" Jul 07 11:00:53.205949 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.205932 2581 scope.go:117] "RemoveContainer" containerID="8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0" Jul 07 11:00:53.206197 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:00:53.206180 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0\": container with ID starting with 8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0 not found: ID does not exist" containerID="8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0" Jul 07 11:00:53.206258 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.206204 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0"} err="failed to get container status \"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0\": rpc error: code = NotFound desc = could not find container \"8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0\": container with ID starting with 8fe12744285ae1a23fca57a45c222c2fdbf9a9a7179b295e36351c525cbf5cd0 not found: ID does not exist" Jul 07 11:00:53.206258 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.206220 2581 scope.go:117] "RemoveContainer" containerID="e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729" Jul 07 11:00:53.206445 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:00:53.206429 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729\": container with ID starting with e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729 not found: ID does not exist" containerID="e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729" Jul 07 11:00:53.206482 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.206448 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729"} err="failed to get container status \"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729\": rpc error: code = NotFound desc = could not find container \"e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729\": container with ID starting with e7d72cb39cd7ebfc82ae5ff49d81b39ae70853498147a30c03f38605f861b729 not found: ID does not exist" Jul 07 11:00:53.235787 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235763 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwcxb\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.235900 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235811 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.235900 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235841 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.235900 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235867 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.235900 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235890 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.236124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235906 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.236124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235941 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.236124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.235985 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.236124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236058 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:00:53.236124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236067 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo\") pod \"c5aaa703-6006-4ab6-a04d-af2ca0540098\" (UID: \"c5aaa703-6006-4ab6-a04d-af2ca0540098\") " Jul 07 11:00:53.236373 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236220 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:00:53.236434 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236408 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:00:53.236533 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236512 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:00:53.236632 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236599 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.236632 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236622 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.236732 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236637 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5aaa703-6006-4ab6-a04d-af2ca0540098-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.236732 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.236650 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.237169 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.237141 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:00:53.238290 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.238262 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:00:53.238412 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.238395 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:00:53.238568 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.238549 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:00:53.238568 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.238559 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb" (OuterVolumeSpecName: "kube-api-access-wwcxb") pod "c5aaa703-6006-4ab6-a04d-af2ca0540098" (UID: "c5aaa703-6006-4ab6-a04d-af2ca0540098"). InnerVolumeSpecName "kube-api-access-wwcxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:00:53.337685 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.337659 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.337685 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.337685 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwcxb\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-kube-api-access-wwcxb\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.337924 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.337694 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.337924 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.337702 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.337924 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.337709 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5aaa703-6006-4ab6-a04d-af2ca0540098-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:00:53.513467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.513432 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 11:00:53.517564 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:53.517536 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-27bc88-openshift-default-68d85dbbcc-mgvmk"] Jul 07 11:00:54.949868 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:54.949835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:55.050227 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.050194 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:55.082078 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.082048 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" path="/var/lib/kubelet/pods/c5aaa703-6006-4ab6-a04d-af2ca0540098/volumes" Jul 07 11:00:55.207591 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.207511 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 11:00:55.207750 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.207729 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" containerID="cri-o://3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab" gracePeriod=30 Jul 07 11:00:55.230594 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.230564 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:55.462984 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.462890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:55.795958 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:55.795864 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:56.093388 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.093305 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:00:56.094020 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.093993 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.094020 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.094021 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.094187 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.094035 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.094187 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.094043 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.094187 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.094139 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.094187 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.094154 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5aaa703-6006-4ab6-a04d-af2ca0540098" containerName="istio-proxy" Jul 07 11:00:56.098615 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.098592 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.101418 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.101394 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5a74a9-openshift-default-dockercfg-c4qq4\"" Jul 07 11:00:56.109304 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.109282 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:00:56.126061 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.126029 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:56.266825 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.266763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.266825 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.266819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.266897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.266986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.267013 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267221 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.267087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267221 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.267134 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267221 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.267166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.267221 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.267210 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njsh\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368086 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.367996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368086 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368086 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368239 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368359 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njsh\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368677 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368677 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368677 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368658 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.368893 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368666 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.369006 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.368982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.370670 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.370651 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.370925 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.370905 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.378152 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.378127 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.378249 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.378129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njsh\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh\") pod \"e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.409683 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.409658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:56.552785 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.552754 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:00:56.554889 ip-10-0-139-204 kubenswrapper[2581]: W0707 11:00:56.554861 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044b307a_34ca_4f7f_a31c_6b15a7240cc5.slice/crio-401689ae6121e629c9ee8ce74901849209d9ab33ec9e4140166ea8a832ba0f97 WatchSource:0}: Error finding container 401689ae6121e629c9ee8ce74901849209d9ab33ec9e4140166ea8a832ba0f97: Status 404 returned error can't find the container with id 401689ae6121e629c9ee8ce74901849209d9ab33ec9e4140166ea8a832ba0f97 Jul 07 11:00:56.557401 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.557369 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:00:56.557499 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.557434 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:00:56.557499 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.557466 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:00:56.606251 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:56.606211 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:57.215197 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:57.215160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerStarted","Data":"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa"} Jul 07 11:00:57.215579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:57.215214 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerStarted","Data":"401689ae6121e629c9ee8ce74901849209d9ab33ec9e4140166ea8a832ba0f97"} Jul 07 11:00:57.236603 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:57.236546 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podStartSLOduration=1.236528254 podStartE2EDuration="1.236528254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:00:57.235223049 +0000 UTC m=+2024.695757817" watchObservedRunningTime="2026-07-07 11:00:57.236528254 +0000 UTC m=+2024.697063000" Jul 07 11:00:57.410586 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:57.410554 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:57.415949 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:57.415918 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:58.220996 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:58.220954 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:58.221990 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:58.221967 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:00:59.160465 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.160397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:59.251998 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.251964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:59.408470 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.408441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:59.728236 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.728191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:59.830520 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.830477 2581 patch_prober.go:28] interesting pod/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 11:00:59.830688 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.830542 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:00:59.831669 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.831637 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:00:59.937215 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:00:59.937184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:00.469290 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.469229 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 11:01:00.613335 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613297 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613335 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613340 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613364 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613405 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613422 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613450 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613516 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkgs8\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613545 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.613593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613589 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert\") pod \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\" (UID: \"51ee4223-35e4-4150-a9b7-44c62ad0c05f\") " Jul 07 11:01:00.614008 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613617 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:00.614008 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613774 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:00.614008 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.613782 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:00.614174 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.614069 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.614174 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.614088 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.614174 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.614103 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.614174 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.614130 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:01:00.614627 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.614598 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data" (OuterVolumeSpecName: "istio-data") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:00.615880 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.615846 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:00.616017 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.615992 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:01:00.616069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.616016 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8" (OuterVolumeSpecName: "kube-api-access-kkgs8") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "kube-api-access-kkgs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:00.616069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.616043 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token" (OuterVolumeSpecName: "istio-token") pod "51ee4223-35e4-4150-a9b7-44c62ad0c05f" (UID: "51ee4223-35e4-4150-a9b7-44c62ad0c05f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:00.714736 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714703 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.714736 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714732 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.714736 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714743 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkgs8\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-kube-api-access-kkgs8\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.714995 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714753 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.714995 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714762 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:00.714995 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:00.714770 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ee4223-35e4-4150-a9b7-44c62ad0c05f-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:01.241145 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.241109 2581 generic.go:356] "Generic (PLEG): container finished" podID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerID="3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab" exitCode=0 Jul 07 11:01:01.241286 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.241164 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" Jul 07 11:01:01.241286 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.241201 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerDied","Data":"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab"} Jul 07 11:01:01.241286 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.241272 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m" event={"ID":"51ee4223-35e4-4150-a9b7-44c62ad0c05f","Type":"ContainerDied","Data":"138a49bafb6e16e37d3261c46cbcf72b06f78453a25838060f82da32e0f85e2d"} Jul 07 11:01:01.241440 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.241310 2581 scope.go:117] "RemoveContainer" containerID="3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab" Jul 07 11:01:01.250429 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.250410 2581 scope.go:117] "RemoveContainer" containerID="f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35" Jul 07 11:01:01.259158 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.259135 2581 scope.go:117] "RemoveContainer" containerID="3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab" Jul 07 11:01:01.259427 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:01.259410 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab\": container with ID starting with 3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab not found: ID does not exist" containerID="3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab" Jul 07 11:01:01.259480 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.259435 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab"} err="failed to get container status \"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab\": rpc error: code = NotFound desc = could not find container \"3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab\": container with ID starting with 3fff9755a97fd3d271eccacb4bb1ff86ff8b98b26e683a4d7fcb3659c7d38bab not found: ID does not exist" Jul 07 11:01:01.259480 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.259452 2581 scope.go:117] "RemoveContainer" containerID="f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35" Jul 07 11:01:01.259691 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:01.259673 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35\": container with ID starting with f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35 not found: ID does not exist" containerID="f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35" Jul 07 11:01:01.259735 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.259696 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35"} err="failed to get container status \"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35\": rpc error: code = NotFound desc = could not find container \"f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35\": container with ID starting with f3afec082d833b7d1f7733935785963d6e07a599de7ae46e2296e1b08283cd35 not found: ID does not exist" Jul 07 11:01:01.264479 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.264454 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 11:01:01.268314 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:01.268294 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-845408-openshift-default-6845fb9f9c-w6m6m"] Jul 07 11:01:03.082083 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:03.082044 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" path="/var/lib/kubelet/pods/51ee4223-35e4-4150-a9b7-44c62ad0c05f/volumes" Jul 07 11:01:03.662027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:03.661980 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:04.126749 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.126712 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:04.395985 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.395903 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:04.396352 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396334 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.396394 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396358 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.396429 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396410 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.396429 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396420 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.396517 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396507 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.396553 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.396522 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ee4223-35e4-4150-a9b7-44c62ad0c05f" containerName="istio-proxy" Jul 07 11:01:04.401155 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.401136 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.404411 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.404389 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-dcb300-openshift-default-dockercfg-dzqxk\"" Jul 07 11:01:04.413827 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.413784 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:04.438099 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.438056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:04.555287 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555460 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555301 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555460 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555373 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555460 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555616 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555616 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555616 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555540 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn224\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555616 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.555737 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.555617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.656945 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.656854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.656945 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.656915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.656950 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.656968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.656998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn224\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657135 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657553 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657482 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657657 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657632 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657842 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657786 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.657842 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657835 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.658014 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.657916 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.660102 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.660076 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.660282 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.660265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.666035 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.666006 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn224\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.666271 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.666254 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token\") pod \"e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.683934 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.683900 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:04.713839 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.713783 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:04.876002 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.875976 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:04.877482 ip-10-0-139-204 kubenswrapper[2581]: W0707 11:01:04.877458 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c62022_908e_496c_9aee_add5ba45dd44.slice/crio-1dbbcda15c4593e615ced8658151843bc5cfe7e7dce2c9781f48d63e4085c0bf WatchSource:0}: Error finding container 1dbbcda15c4593e615ced8658151843bc5cfe7e7dce2c9781f48d63e4085c0bf: Status 404 returned error can't find the container with id 1dbbcda15c4593e615ced8658151843bc5cfe7e7dce2c9781f48d63e4085c0bf Jul 07 11:01:04.879733 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.879696 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:04.879868 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.879765 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:04.879868 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:04.879811 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:05.264533 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:05.264429 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerStarted","Data":"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7"} Jul 07 11:01:05.264533 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:05.264487 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerStarted","Data":"1dbbcda15c4593e615ced8658151843bc5cfe7e7dce2c9781f48d63e4085c0bf"} Jul 07 11:01:05.288032 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:05.287981 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podStartSLOduration=1.2879604310000001 podStartE2EDuration="1.287960431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:01:05.286310842 +0000 UTC m=+2032.746845609" watchObservedRunningTime="2026-07-07 11:01:05.287960431 +0000 UTC m=+2032.748495177" Jul 07 11:01:05.714621 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:05.714584 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:05.728429 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:05.728360 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:06.269231 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:06.269204 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:06.270363 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:06.270345 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:07.290591 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:07.290557 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:07.383182 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:07.383143 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:07.482706 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:07.482661 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:07.791403 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:07.791369 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:07.891634 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:07.891580 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:11.730587 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:11.730543 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:12.255685 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:12.255652 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:12.355778 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:12.355728 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:13.223476 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.223413 2581 patch_prober.go:28] interesting pod/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 11:01:13.223943 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.223536 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 11:01:13.310246 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.310209 2581 generic.go:356] "Generic (PLEG): container finished" podID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerID="d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa" exitCode=0 Jul 07 11:01:13.310431 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.310285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerDied","Data":"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa"} Jul 07 11:01:13.310720 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.310703 2581 scope.go:117] "RemoveContainer" containerID="d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa" Jul 07 11:01:13.311756 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.311723 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:13.311880 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.311787 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:13.311880 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:13.311846 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:14.318927 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:14.318886 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerStarted","Data":"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0"} Jul 07 11:01:14.410753 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:14.410718 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:14.410950 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:14.410911 2581 patch_prober.go:28] interesting pod/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 11:01:14.411007 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:14.410985 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 11:01:15.410389 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:15.410353 2581 patch_prober.go:28] interesting pod/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 11:01:15.410783 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:15.410409 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 11:01:16.410694 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:16.410648 2581 patch_prober.go:28] interesting pod/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 11:01:16.411203 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:16.410714 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 11:01:17.410407 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:17.410367 2581 patch_prober.go:28] interesting pod/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 11:01:17.410609 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:17.410432 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 11:01:18.414244 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:18.414209 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:18.414906 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:18.414561 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:18.415161 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:18.415141 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:19.356215 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.356106 2581 generic.go:356] "Generic (PLEG): container finished" podID="05c62022-908e-496c-9aee-add5ba45dd44" containerID="55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7" exitCode=0 Jul 07 11:01:19.356409 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.356180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerDied","Data":"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7"} Jul 07 11:01:19.356719 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.356705 2581 scope.go:117] "RemoveContainer" containerID="55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7" Jul 07 11:01:19.357653 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.357628 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:19.357712 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.357692 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:19.357750 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.357723 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:19.862179 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:19.862146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:20.281918 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.281882 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:20.364279 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.364244 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerStarted","Data":"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b"} Jul 07 11:01:20.408313 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.408273 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:20.508742 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.508705 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:20.699716 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.699664 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:20.714149 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.714119 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:20.715113 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.715092 2581 patch_prober.go:28] interesting pod/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 11:01:20.715200 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:20.715137 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 11:01:21.269504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:21.269473 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:21.714646 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:21.714611 2581 patch_prober.go:28] interesting pod/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 11:01:21.714855 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:21.714665 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 11:01:22.714918 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:22.714881 2581 patch_prober.go:28] interesting pod/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 11:01:22.715304 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:22.714939 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 11:01:23.714191 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:23.714149 2581 patch_prober.go:28] interesting pod/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 11:01:23.714412 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:23.714217 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 11:01:24.718337 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:24.718300 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:24.719362 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:24.719342 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:25.886442 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:25.886408 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:26.556882 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:26.556843 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:26.635168 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:26.635136 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:26.789132 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:26.789097 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:01:26.789424 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:26.789371 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" containerID="cri-o://e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0" gracePeriod=30 Jul 07 11:01:26.866261 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:26.866162 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.042062 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.042022 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.481621 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.481579 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.615244 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.615197 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.716521 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.716484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.818714 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.818544 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:27.936670 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:27.936632 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:32.047182 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.047155 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:32.155141 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155107 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155333 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155172 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155333 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155197 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155333 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155306 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155335 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155355 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155391 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155445 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155503 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8njsh\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh\") pod \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\" (UID: \"044b307a-34ca-4f7f-a31c-6b15a7240cc5\") " Jul 07 11:01:32.155760 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155617 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:32.155760 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155638 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:32.155918 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155763 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:32.155986 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155944 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.155986 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155963 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.156095 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.155996 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.156095 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.156062 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:01:32.156320 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.156291 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data" (OuterVolumeSpecName: "istio-data") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:32.157843 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.157769 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh" (OuterVolumeSpecName: "kube-api-access-8njsh") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "kube-api-access-8njsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:32.157943 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.157895 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token" (OuterVolumeSpecName: "istio-token") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:32.158082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.158055 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:01:32.158160 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.158094 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "044b307a-34ca-4f7f-a31c-6b15a7240cc5" (UID: "044b307a-34ca-4f7f-a31c-6b15a7240cc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:32.257467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257414 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.257467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257459 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.257467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257469 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8njsh\" (UniqueName: \"kubernetes.io/projected/044b307a-34ca-4f7f-a31c-6b15a7240cc5-kube-api-access-8njsh\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.257467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257480 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.257467 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257488 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.257777 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.257496 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/044b307a-34ca-4f7f-a31c-6b15a7240cc5-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:32.424851 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.424746 2581 generic.go:356] "Generic (PLEG): container finished" podID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerID="e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0" exitCode=0 Jul 07 11:01:32.424851 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.424789 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerDied","Data":"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0"} Jul 07 11:01:32.424851 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.424813 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" Jul 07 11:01:32.425085 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.424855 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw" event={"ID":"044b307a-34ca-4f7f-a31c-6b15a7240cc5","Type":"ContainerDied","Data":"401689ae6121e629c9ee8ce74901849209d9ab33ec9e4140166ea8a832ba0f97"} Jul 07 11:01:32.425085 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.424887 2581 scope.go:117] "RemoveContainer" containerID="e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0" Jul 07 11:01:32.434991 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.434973 2581 scope.go:117] "RemoveContainer" containerID="d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa" Jul 07 11:01:32.443009 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.442987 2581 scope.go:117] "RemoveContainer" containerID="e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0" Jul 07 11:01:32.443270 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:32.443251 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0\": container with ID starting with e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0 not found: ID does not exist" containerID="e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0" Jul 07 11:01:32.443315 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.443278 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0"} err="failed to get container status \"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0\": rpc error: code = NotFound desc = could not find container \"e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0\": container with ID starting with e8dbc07536d73a75b00f34ff235beeca24ed06f5e3e2d218f052173c3f8960b0 not found: ID does not exist" Jul 07 11:01:32.443315 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.443295 2581 scope.go:117] "RemoveContainer" containerID="d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa" Jul 07 11:01:32.443513 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:32.443495 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa\": container with ID starting with d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa not found: ID does not exist" containerID="d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa" Jul 07 11:01:32.443558 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.443518 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa"} err="failed to get container status \"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa\": rpc error: code = NotFound desc = could not find container \"d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa\": container with ID starting with d9337e49e39f19d9ef5009f92e4f8482ff0c9d56e562144577ec8541d1012ffa not found: ID does not exist" Jul 07 11:01:32.448822 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.448784 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:01:32.452380 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:32.452359 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5a74a9-openshift-default-5d49bdf45c-9hrkw"] Jul 07 11:01:33.080099 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.080065 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" path="/var/lib/kubelet/pods/044b307a-34ca-4f7f-a31c-6b15a7240cc5/volumes" Jul 07 11:01:33.597202 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.597169 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:33.669385 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.669348 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:33.853914 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.853777 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:33.854103 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.854076 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" containerID="cri-o://2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b" gracePeriod=30 Jul 07 11:01:33.883978 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:33.883931 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:34.094853 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.094824 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:34.433247 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.433215 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:34.819933 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.819852 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:01:34.820268 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820254 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.820311 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820269 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.820311 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820285 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.820311 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820291 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.820400 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820360 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.820400 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.820367 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="044b307a-34ca-4f7f-a31c-6b15a7240cc5" containerName="istio-proxy" Jul 07 11:01:34.823420 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.823404 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.827990 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.827959 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1273de-openshift-default-dockercfg-7b884\"" Jul 07 11:01:34.860116 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.860085 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:01:34.955694 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.955660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:34.987504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987693 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987693 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987629 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987693 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987885 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987885 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987748 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987885 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987885 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g2br\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:34.987885 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:34.987854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.060957 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.060917 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:35.089389 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089389 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089344 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089604 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089604 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089418 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089604 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089771 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089771 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089697 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089771 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g2br\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.089771 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089737 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.090030 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.090030 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.089821 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.090139 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.090112 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.090205 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.090180 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.090327 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.090312 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.091949 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.091911 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.092408 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.092383 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.099672 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.099648 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.100032 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.099755 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g2br\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br\") pod \"e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.133962 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.133926 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:35.279431 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.279402 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:01:35.280772 ip-10-0-139-204 kubenswrapper[2581]: W0707 11:01:35.280746 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6c7b59_cae6_49d9_bbac_5fc886ec158f.slice/crio-82cbba12f10b96e446f7d0a56f461f82c7b9590338974ab32ad5d83e11897556 WatchSource:0}: Error finding container 82cbba12f10b96e446f7d0a56f461f82c7b9590338974ab32ad5d83e11897556: Status 404 returned error can't find the container with id 82cbba12f10b96e446f7d0a56f461f82c7b9590338974ab32ad5d83e11897556 Jul 07 11:01:35.283259 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.283219 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:35.283358 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.283320 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:35.283404 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.283372 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:35.443606 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.443566 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerStarted","Data":"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027"} Jul 07 11:01:35.443787 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.443649 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerStarted","Data":"82cbba12f10b96e446f7d0a56f461f82c7b9590338974ab32ad5d83e11897556"} Jul 07 11:01:35.469949 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:35.469885 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podStartSLOduration=1.469863203 podStartE2EDuration="1.469863203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:01:35.466914848 +0000 UTC m=+2062.927449620" watchObservedRunningTime="2026-07-07 11:01:35.469863203 +0000 UTC m=+2062.930397949" Jul 07 11:01:36.135138 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:36.135098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:36.140045 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:36.140022 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:36.449519 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:36.449436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:36.450331 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:36.450314 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:37.402854 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:37.402732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:37.492411 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:37.492382 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:37.564426 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:37.564393 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:37.888731 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:37.888677 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:37.989417 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:37.989376 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:38.090143 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:38.090101 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:39.112235 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.112213 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:39.130694 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130667 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.130845 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130706 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.130845 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130726 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.130845 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130756 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.130845 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130788 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.131071 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130874 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn224\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.131071 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130906 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.131071 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.130967 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.131071 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131010 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy\") pod \"05c62022-908e-496c-9aee-add5ba45dd44\" (UID: \"05c62022-908e-496c-9aee-add5ba45dd44\") " Jul 07 11:01:39.131071 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131020 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:39.131322 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131078 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:39.131322 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131143 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:01:39.131464 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131443 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c62022-908e-496c-9aee-add5ba45dd44-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.131658 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131469 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.131658 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131467 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:39.131658 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131484 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.131964 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.131938 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data" (OuterVolumeSpecName: "istio-data") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:39.133299 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.133271 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224" (OuterVolumeSpecName: "kube-api-access-zn224") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "kube-api-access-zn224". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:39.133419 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.133374 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:01:39.133721 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.133703 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token" (OuterVolumeSpecName: "istio-token") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:01:39.133880 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.133854 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05c62022-908e-496c-9aee-add5ba45dd44" (UID: "05c62022-908e-496c-9aee-add5ba45dd44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:01:39.232296 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232261 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn224\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-kube-api-access-zn224\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.232296 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232292 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c62022-908e-496c-9aee-add5ba45dd44-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.232296 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232302 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.232535 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232310 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.232535 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232318 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c62022-908e-496c-9aee-add5ba45dd44-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.232535 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.232325 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c62022-908e-496c-9aee-add5ba45dd44-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:01:39.467751 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.467662 2581 generic.go:356] "Generic (PLEG): container finished" podID="05c62022-908e-496c-9aee-add5ba45dd44" containerID="2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b" exitCode=0 Jul 07 11:01:39.467751 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.467712 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" Jul 07 11:01:39.467957 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.467749 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerDied","Data":"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b"} Jul 07 11:01:39.467957 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.467817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g" event={"ID":"05c62022-908e-496c-9aee-add5ba45dd44","Type":"ContainerDied","Data":"1dbbcda15c4593e615ced8658151843bc5cfe7e7dce2c9781f48d63e4085c0bf"} Jul 07 11:01:39.467957 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.467843 2581 scope.go:117] "RemoveContainer" containerID="2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b" Jul 07 11:01:39.477420 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.477402 2581 scope.go:117] "RemoveContainer" containerID="55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7" Jul 07 11:01:39.486446 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.486427 2581 scope.go:117] "RemoveContainer" containerID="2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b" Jul 07 11:01:39.486715 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:39.486694 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b\": container with ID starting with 2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b not found: ID does not exist" containerID="2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b" Jul 07 11:01:39.486773 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.486722 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b"} err="failed to get container status \"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b\": rpc error: code = NotFound desc = could not find container \"2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b\": container with ID starting with 2f501a1f701c83f0ed1ef58966ba8fab6819e502a3409d5532e4996461a73b1b not found: ID does not exist" Jul 07 11:01:39.486773 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.486739 2581 scope.go:117] "RemoveContainer" containerID="55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7" Jul 07 11:01:39.486982 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:01:39.486964 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7\": container with ID starting with 55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7 not found: ID does not exist" containerID="55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7" Jul 07 11:01:39.487025 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.486988 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7"} err="failed to get container status \"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7\": rpc error: code = NotFound desc = could not find container \"55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7\": container with ID starting with 55bece991119f87f4ed7c90404f098d2834210d528621709c23b0accd9339ff7 not found: ID does not exist" Jul 07 11:01:39.494330 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.494308 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:39.498333 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:39.498313 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-dcb300-openshift-default-7d67f6ffcc-psq8g"] Jul 07 11:01:41.079776 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:41.079740 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c62022-908e-496c-9aee-add5ba45dd44" path="/var/lib/kubelet/pods/05c62022-908e-496c-9aee-add5ba45dd44/volumes" Jul 07 11:01:41.941337 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:41.941281 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:42.243559 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.243469 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:42.524264 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.524181 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:01:42.524856 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.524831 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:01:42.524856 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.524857 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:01:42.525072 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.524886 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:01:42.525072 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.524895 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:01:42.525072 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.525037 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:01:42.528006 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.527989 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.530695 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.530673 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2a3633-openshift-default-dockercfg-g5cn7\"" Jul 07 11:01:42.538077 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.538053 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:01:42.561393 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561363 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561507 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561782 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561782 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561782 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.561782 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.561702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsp8q\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.563318 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.563290 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:42.662228 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662228 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662232 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662354 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662504 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662826 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsp8q\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662826 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662826 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662726 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662986 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662846 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.662986 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.662912 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.663218 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.663198 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.664734 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.664707 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.665177 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.665157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.683083 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.683048 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.691857 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.691831 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsp8q\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q\") pod \"e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.735682 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.735641 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:42.839729 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.839638 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:42.991931 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.991904 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:01:42.993907 ip-10-0-139-204 kubenswrapper[2581]: W0707 11:01:42.993883 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf598ba31_2454_4255_905b_1bd0ffac5bd4.slice/crio-fcd068d56d7c431b9552b8a1662e81fe1bbb894497b3f6c9ad47e9490a7b877e WatchSource:0}: Error finding container fcd068d56d7c431b9552b8a1662e81fe1bbb894497b3f6c9ad47e9490a7b877e: Status 404 returned error can't find the container with id fcd068d56d7c431b9552b8a1662e81fe1bbb894497b3f6c9ad47e9490a7b877e Jul 07 11:01:42.996127 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.996098 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:42.996180 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.996160 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:42.996221 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:42.996190 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:43.489501 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:43.489463 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerStarted","Data":"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302"} Jul 07 11:01:43.490027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:43.489525 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerStarted","Data":"fcd068d56d7c431b9552b8a1662e81fe1bbb894497b3f6c9ad47e9490a7b877e"} Jul 07 11:01:43.513024 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:43.512961 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podStartSLOduration=1.512937398 podStartE2EDuration="1.512937398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:01:43.510579763 +0000 UTC m=+2070.971114516" watchObservedRunningTime="2026-07-07 11:01:43.512937398 +0000 UTC m=+2070.973472147" Jul 07 11:01:43.840137 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:43.840048 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:44.846126 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:44.846101 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:45.435219 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.435175 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:45.500709 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.500674 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:45.501966 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.501935 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:01:45.592619 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.592582 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:45.693661 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.693561 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:45.796255 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:45.796185 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:46.229645 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:46.229604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:46.329593 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:46.329556 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:49.941882 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:49.941846 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:52.490986 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:52.490946 2581 patch_prober.go:28] interesting pod/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 11:01:52.491529 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:52.491032 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 11:01:53.572140 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.572059 2581 generic.go:356] "Generic (PLEG): container finished" podID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerID="1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027" exitCode=0 Jul 07 11:01:53.572140 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.572130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerDied","Data":"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027"} Jul 07 11:01:53.572579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.572550 2581 scope.go:117] "RemoveContainer" containerID="1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027" Jul 07 11:01:53.573552 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.573526 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:53.573816 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.573592 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:53.573816 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:53.573633 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:01:54.580562 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:54.580521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerStarted","Data":"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577"} Jul 07 11:01:55.134960 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:55.134921 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:55.135347 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:55.135259 2581 patch_prober.go:28] interesting pod/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 11:01:55.135515 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:55.135492 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 11:01:56.135062 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:56.135021 2581 patch_prober.go:28] interesting pod/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 11:01:56.135659 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:56.135626 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 11:01:57.134987 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:57.134511 2581 patch_prober.go:28] interesting pod/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 11:01:57.134987 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:57.134581 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 11:01:58.135118 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:58.135081 2581 patch_prober.go:28] interesting pod/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 11:01:58.135427 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:58.135388 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 11:01:58.739490 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:58.739022 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:58.836768 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:58.836733 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:01:59.138689 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:59.138651 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:59.139178 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:59.139108 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:01:59.139770 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:01:59.139749 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:02:00.502214 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.502178 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:00.502628 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.502242 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:00.616276 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.616244 2581 generic.go:356] "Generic (PLEG): container finished" podID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerID="aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302" exitCode=0 Jul 07 11:02:00.616452 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.616318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerDied","Data":"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302"} Jul 07 11:02:00.616841 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.616824 2581 scope.go:117] "RemoveContainer" containerID="aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302" Jul 07 11:02:00.617833 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.617788 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:02:00.617943 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.617885 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:02:00.617943 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:00.617936 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 11:02:01.624545 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:01.624502 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerStarted","Data":"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59"} Jul 07 11:02:01.840109 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:01.840067 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:02:01.840313 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:01.840290 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:01.840360 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:01.840341 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:02.840549 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:02.840505 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:02.841214 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:02.841175 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:03.840652 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:03.840610 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:03.841182 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:03.840893 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:04.840369 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:04.840316 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:04.840613 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:04.840389 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:05.840306 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:05.840253 2581 patch_prober.go:28] interesting pod/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 11:02:05.840856 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:05.840326 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 11:02:06.844080 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:06.844048 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:02:06.844475 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:06.844410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:02:06.845045 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:06.845029 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:02:07.077705 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:07.077668 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:02:07.178964 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:07.178925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:02:14.710257 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:14.710206 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:02:35.518682 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:35.518636 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:02:46.426644 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:46.426604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:02:46.486894 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:02:46.486858 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:03:13.975137 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:03:13.975041 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:03:35.324692 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:03:35.324657 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:04:00.473441 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:04:00.473393 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:04:31.738433 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:04:31.738397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:04:32.650369 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:04:32.650335 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:04:52.079025 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:04:52.078982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:05:14.895015 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:05:14.894968 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:05:46.744255 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:05:46.744214 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:05:47.628077 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:05:47.628046 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:05:53.357140 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:05:53.357108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:06:12.972936 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:06:12.972842 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:06:13.077551 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:06:13.077516 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:06:19.914989 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:06:19.914948 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:06:59.629615 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:06:59.629570 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:06:59.729927 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:06:59.729890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:06.357760 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:06.357719 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:07.376513 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:07.376473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:07.807313 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:07.807228 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:07.918108 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:07.918071 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:08.475924 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:08.475890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:08.882847 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:08.882815 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:08.983111 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:08.983075 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:09.085403 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:09.085360 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:09.306175 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:09.306080 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:13.950286 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:13.950253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:50.925966 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:50.925924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:51.152671 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.152644 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:07:51.152920 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.152896 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" containerID="cri-o://57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577" gracePeriod=30 Jul 07 11:07:51.177637 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.177548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:51.399978 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.399945 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:51.558529 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.558437 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:51.976082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:51.976039 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:52.075253 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:52.075217 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:52.238604 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:52.238504 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:56.416859 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.416779 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:07:56.495184 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.495151 2581 generic.go:356] "Generic (PLEG): container finished" podID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerID="57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577" exitCode=0 Jul 07 11:07:56.495345 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.495205 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" Jul 07 11:07:56.495345 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.495240 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerDied","Data":"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577"} Jul 07 11:07:56.495345 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.495302 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx" event={"ID":"8c6c7b59-cae6-49d9-bbac-5fc886ec158f","Type":"ContainerDied","Data":"82cbba12f10b96e446f7d0a56f461f82c7b9590338974ab32ad5d83e11897556"} Jul 07 11:07:56.495345 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.495333 2581 scope.go:117] "RemoveContainer" containerID="57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577" Jul 07 11:07:56.504195 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.504172 2581 scope.go:117] "RemoveContainer" containerID="1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027" Jul 07 11:07:56.512410 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.512379 2581 scope.go:117] "RemoveContainer" containerID="57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577" Jul 07 11:07:56.512681 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:07:56.512662 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577\": container with ID starting with 57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577 not found: ID does not exist" containerID="57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577" Jul 07 11:07:56.512728 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.512692 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577"} err="failed to get container status \"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577\": rpc error: code = NotFound desc = could not find container \"57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577\": container with ID starting with 57250c10319734df33aa0b854f022e95a1ae4cee2d51b722d4313cb872e15577 not found: ID does not exist" Jul 07 11:07:56.512728 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.512711 2581 scope.go:117] "RemoveContainer" containerID="1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027" Jul 07 11:07:56.513022 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:07:56.512998 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027\": container with ID starting with 1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027 not found: ID does not exist" containerID="1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027" Jul 07 11:07:56.513094 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.513031 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027"} err="failed to get container status \"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027\": rpc error: code = NotFound desc = could not find container \"1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027\": container with ID starting with 1aa1da8b72e05f840e2cbf481213eee2a9c3a6587db591b37e4f687de129f027 not found: ID does not exist" Jul 07 11:07:56.523960 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.523939 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524047 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.523969 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g2br\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524047 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524023 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524118 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524089 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524157 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524140 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524190 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524166 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524227 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524202 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524261 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524240 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524297 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524279 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data\") pod \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\" (UID: \"8c6c7b59-cae6-49d9-bbac-5fc886ec158f\") " Jul 07 11:07:56.524719 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524676 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:07:56.524848 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.524810 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:07:56.525133 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.525021 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:07:56.525133 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.525036 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:07:56.525898 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.525872 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:07:56.526527 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.526497 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:07:56.526852 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.526831 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br" (OuterVolumeSpecName: "kube-api-access-5g2br") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "kube-api-access-5g2br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:07:56.526944 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.526871 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:07:56.527117 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.527098 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c6c7b59-cae6-49d9-bbac-5fc886ec158f" (UID: "8c6c7b59-cae6-49d9-bbac-5fc886ec158f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:07:56.625655 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625613 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625655 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625647 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625655 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625656 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625655 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625666 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625955 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625674 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625955 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625682 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g2br\" (UniqueName: \"kubernetes.io/projected/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-kube-api-access-5g2br\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625955 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625692 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625955 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625700 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.625955 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.625707 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6c7b59-cae6-49d9-bbac-5fc886ec158f-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:07:56.831869 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.831839 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:07:56.846098 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:56.846069 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1273de-openshift-default-55f8f7cb75-vbcjx"] Jul 07 11:07:57.080347 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:57.080248 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" path="/var/lib/kubelet/pods/8c6c7b59-cae6-49d9-bbac-5fc886ec158f/volumes" Jul 07 11:07:57.845454 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:57.845410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:57.945620 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:57.945591 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:58.082992 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:58.082955 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:07:58.083210 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:58.083186 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" containerID="cri-o://c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59" gracePeriod=30 Jul 07 11:07:58.105420 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:58.105338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:58.450293 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:58.450260 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:07:58.778278 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:07:58.778186 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5797c5c47b-pfvm6_513172af-ef5f-47b1-a985-6130e64a6451/authorino/0.log" Jul 07 11:08:02.951961 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:02.951926 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:08:03.288861 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.288825 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-swm8t_ff3a74ee-75bb-4ad1-ac14-733562b9197d/manager/0.log" Jul 07 11:08:03.345638 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.345605 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:08:03.412400 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.412365 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lvrtz_498a8dba-7f57-449d-a0c3-051d6b00e683/manager/0.log" Jul 07 11:08:03.495339 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495301 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsp8q\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495356 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495380 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495402 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495444 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495460 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495532 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495530 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495907 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495564 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495907 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495593 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket\") pod \"f598ba31-2454-4255-905b-1bd0ffac5bd4\" (UID: \"f598ba31-2454-4255-905b-1bd0ffac5bd4\") " Jul 07 11:08:03.495907 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495833 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:08:03.496065 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.495927 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:08:03.496065 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.496013 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.496065 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.496046 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:08:03.496229 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.496205 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:08:03.497013 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.496988 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:08:03.497965 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.497940 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:08:03.498065 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.498045 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:08:03.498254 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.498233 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q" (OuterVolumeSpecName: "kube-api-access-xsp8q") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "kube-api-access-xsp8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:08:03.498254 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.498239 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f598ba31-2454-4255-905b-1bd0ffac5bd4" (UID: "f598ba31-2454-4255-905b-1bd0ffac5bd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 11:08:03.537192 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.537159 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7fdd849877-9kxz9_a6768bfd-1a7e-49a1-853b-4140138e2ade/maas-api/0.log" Jul 07 11:08:03.540715 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.540686 2581 generic.go:356] "Generic (PLEG): container finished" podID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerID="c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59" exitCode=0 Jul 07 11:08:03.540894 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.540749 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" Jul 07 11:08:03.540894 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.540769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerDied","Data":"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59"} Jul 07 11:08:03.540894 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.540835 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp" event={"ID":"f598ba31-2454-4255-905b-1bd0ffac5bd4","Type":"ContainerDied","Data":"fcd068d56d7c431b9552b8a1662e81fe1bbb894497b3f6c9ad47e9490a7b877e"} Jul 07 11:08:03.540894 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.540859 2581 scope.go:117] "RemoveContainer" containerID="c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59" Jul 07 11:08:03.550873 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.550854 2581 scope.go:117] "RemoveContainer" containerID="aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302" Jul 07 11:08:03.560069 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.560047 2581 scope.go:117] "RemoveContainer" containerID="c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59" Jul 07 11:08:03.560317 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:08:03.560298 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59\": container with ID starting with c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59 not found: ID does not exist" containerID="c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59" Jul 07 11:08:03.560364 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.560327 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59"} err="failed to get container status \"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59\": rpc error: code = NotFound desc = could not find container \"c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59\": container with ID starting with c66127821f0c8614bee46c93e3fbc3876dc2aa19d32547a9bcd64861642e9b59 not found: ID does not exist" Jul 07 11:08:03.560364 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.560346 2581 scope.go:117] "RemoveContainer" containerID="aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302" Jul 07 11:08:03.560572 ip-10-0-139-204 kubenswrapper[2581]: E0707 11:08:03.560557 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302\": container with ID starting with aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302 not found: ID does not exist" containerID="aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302" Jul 07 11:08:03.560616 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.560575 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302"} err="failed to get container status \"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302\": rpc error: code = NotFound desc = could not find container \"aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302\": container with ID starting with aef608a9e4e48dfb1d728cebf7543cb80f7fb9560fdf71f958675004f1196302 not found: ID does not exist" Jul 07 11:08:03.565541 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.565514 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:08:03.571134 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.571115 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2a3633-openshift-default-7b498c8df5-h4mvp"] Jul 07 11:08:03.596945 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596921 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-token\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.596945 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596942 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-podinfo\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596951 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-workload-certs\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596962 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-credential-socket\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596971 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsp8q\" (UniqueName: \"kubernetes.io/projected/f598ba31-2454-4255-905b-1bd0ffac5bd4-kube-api-access-xsp8q\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596979 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-data\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596987 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f598ba31-2454-4255-905b-1bd0ffac5bd4-istio-envoy\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.597082 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.596995 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f598ba31-2454-4255-905b-1bd0ffac5bd4-istiod-ca-cert\") on node \"ip-10-0-139-204.ec2.internal\" DevicePath \"\"" Jul 07 11:08:03.902395 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:03.902367 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bcd9f7d5c-lbhrl_182ce1ca-b7ca-4886-bacb-7d69bbd5d37c/manager/0.log" Jul 07 11:08:04.143358 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:04.143320 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-74927_6c27825c-ada8-4a89-9734-b39cb4622193/manager/0.log" Jul 07 11:08:04.392982 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:04.392948 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-586fcf4d5b-f94lq_4631b4a6-fb49-43eb-bb12-ce5590455817/copy-manifests/0.log" Jul 07 11:08:04.400676 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:04.400642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-586fcf4d5b-f94lq_4631b4a6-fb49-43eb-bb12-ce5590455817/manager/0.log" Jul 07 11:08:04.633971 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:04.633936 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8tjtm_e78edd62-7446-4726-8ba6-7236de296c87/postgres/0.log" Jul 07 11:08:05.079664 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:05.079629 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" path="/var/lib/kubelet/pods/f598ba31-2454-4255-905b-1bd0ffac5bd4/volumes" Jul 07 11:08:05.942521 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:05.942484 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5797c5c47b-pfvm6_513172af-ef5f-47b1-a985-6130e64a6451/authorino/0.log" Jul 07 11:08:06.065568 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:06.065532 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tqlts_4c890e0f-6fd9-46ce-8c01-4f52796c6637/manager/0.log" Jul 07 11:08:06.184833 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:06.184781 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dtstk_952907dc-4a8b-4996-bbcf-5bd1ee3590a6/manager/0.log" Jul 07 11:08:06.292476 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:06.292391 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kv8xd_abf3b900-88a7-4506-84e7-d708564acbca/kuadrant-console-plugin/0.log" Jul 07 11:08:06.419487 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:06.419443 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-x4dg6_e3d71b29-d61e-48ce-a94a-0f18c5e95ea1/registry-server/0.log" Jul 07 11:08:06.788384 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:06.788341 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4hdbt_404d5b52-f916-44b0-bc74-558ad98ad138/manager/0.log" Jul 07 11:08:07.342422 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:07.342395 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4f4t7h_95b07372-b3d8-4bf5-b464-ca2746b7dc85/istio-proxy/0.log" Jul 07 11:08:07.686632 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:07.686606 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c48467d48-ml746_d8825205-6632-4ee8-b4e7-93a2f856f0eb/kube-auth-proxy/0.log" Jul 07 11:08:07.810552 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:07.810521 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/1.log" Jul 07 11:08:07.931626 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:07.931591 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fkfld_30b658d7-ce36-48e9-a904-a8cc598deb70/payload-pre-processing/0.log" Jul 07 11:08:08.159050 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:08.159020 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68d68d4cdf-ghwnt_438503da-c32e-49be-9296-4c1beb56f484/router/0.log" Jul 07 11:08:08.509415 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:08.509327 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-vhh96_5c0aa737-1413-40a9-ae6d-343da9c2c6a9/main/0.log" Jul 07 11:08:08.853086 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:08.852999 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbztdh5_af42eec7-1175-4250-b829-2430c308c0e7/main/0.log" Jul 07 11:08:13.247934 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:13.247895 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:08:13.322169 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:13.322123 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" limit=1024 Jul 07 11:08:16.484391 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:16.484357 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h9wvp_a2087d4e-9d80-4d94-b537-077d2901c6fa/global-pull-secret-syncer/0.log" Jul 07 11:08:16.548579 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:16.548548 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p7s6m_92bdf557-5f55-49e0-9c91-eb18897f1e92/konnectivity-agent/0.log" Jul 07 11:08:16.698629 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:16.698599 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-204.ec2.internal_9585bff18ee532ee27e6d3ba6f76cb3f/haproxy/0.log" Jul 07 11:08:21.442025 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.441985 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5797c5c47b-pfvm6_513172af-ef5f-47b1-a985-6130e64a6451/authorino/0.log" Jul 07 11:08:21.476724 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.476693 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tqlts_4c890e0f-6fd9-46ce-8c01-4f52796c6637/manager/0.log" Jul 07 11:08:21.521017 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.520987 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dtstk_952907dc-4a8b-4996-bbcf-5bd1ee3590a6/manager/0.log" Jul 07 11:08:21.569214 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.569189 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kv8xd_abf3b900-88a7-4506-84e7-d708564acbca/kuadrant-console-plugin/0.log" Jul 07 11:08:21.622075 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.622044 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-x4dg6_e3d71b29-d61e-48ce-a94a-0f18c5e95ea1/registry-server/0.log" Jul 07 11:08:21.910495 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:21.910463 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4hdbt_404d5b52-f916-44b0-bc74-558ad98ad138/manager/0.log" Jul 07 11:08:23.919025 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:23.918992 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-75ct4_2c68b91c-e816-4609-8162-3fc148c2fb4e/kube-state-metrics/0.log" Jul 07 11:08:23.951758 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:23.951721 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-75ct4_2c68b91c-e816-4609-8162-3fc148c2fb4e/kube-rbac-proxy-main/0.log" Jul 07 11:08:23.981936 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:23.981902 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-75ct4_2c68b91c-e816-4609-8162-3fc148c2fb4e/kube-rbac-proxy-self/0.log" Jul 07 11:08:24.020386 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:24.020354 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b9f9468d6-8g9hd_cfa81a65-6f6e-450c-9567-501f3d844a85/metrics-server/0.log" Jul 07 11:08:24.250649 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:24.250554 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbqq9_be115682-2260-46a2-a57e-0e23166e00f7/node-exporter/0.log" Jul 07 11:08:24.272329 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:24.272303 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbqq9_be115682-2260-46a2-a57e-0e23166e00f7/kube-rbac-proxy/0.log" Jul 07 11:08:24.298530 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:24.298505 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mbqq9_be115682-2260-46a2-a57e-0e23166e00f7/init-textfile/0.log" Jul 07 11:08:26.372647 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.372611 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg"] Jul 07 11:08:26.373062 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373047 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" Jul 07 11:08:26.373107 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373064 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" Jul 07 11:08:26.373107 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373075 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.373107 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373080 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.373107 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373097 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.373107 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373102 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.373267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373165 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" Jul 07 11:08:26.373267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373175 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c62022-908e-496c-9aee-add5ba45dd44" containerName="istio-proxy" Jul 07 11:08:26.373267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373181 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6c7b59-cae6-49d9-bbac-5fc886ec158f" containerName="istio-proxy" Jul 07 11:08:26.373267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373187 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.373267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.373193 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f598ba31-2454-4255-905b-1bd0ffac5bd4" containerName="istio-proxy" Jul 07 11:08:26.376588 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.376567 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.379512 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.379494 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rgbf7\"/\"kube-root-ca.crt\"" Jul 07 11:08:26.379662 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.379644 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rgbf7\"/\"openshift-service-ca.crt\"" Jul 07 11:08:26.380595 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.380582 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rgbf7\"/\"default-dockercfg-9prl2\"" Jul 07 11:08:26.387170 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.387145 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg"] Jul 07 11:08:26.426453 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.426415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-podres\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.426618 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.426482 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-sys\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.426618 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.426569 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs69x\" (UniqueName: \"kubernetes.io/projected/caf0918b-f331-491a-a55f-3ceb52befa20-kube-api-access-vs69x\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.426618 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.426615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-lib-modules\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.426755 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.426654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-proc\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.527827 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527760 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-proc\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527851 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-podres\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527887 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-proc\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-sys\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527937 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-sys\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528027 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.527996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs69x\" (UniqueName: \"kubernetes.io/projected/caf0918b-f331-491a-a55f-3ceb52befa20-kube-api-access-vs69x\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528292 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.528037 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-lib-modules\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528292 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.528001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-podres\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.528292 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.528163 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/caf0918b-f331-491a-a55f-3ceb52befa20-lib-modules\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.537124 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.537100 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs69x\" (UniqueName: \"kubernetes.io/projected/caf0918b-f331-491a-a55f-3ceb52befa20-kube-api-access-vs69x\") pod \"perf-node-gather-daemonset-mfcsg\" (UID: \"caf0918b-f331-491a-a55f-3ceb52befa20\") " pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.686980 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.686889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:26.827813 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:26.827766 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg"] Jul 07 11:08:26.829743 ip-10-0-139-204 kubenswrapper[2581]: W0707 11:08:26.829714 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcaf0918b_f331_491a_a55f_3ceb52befa20.slice/crio-af5ff6b4b0d75d4c2b37bd7595e9bf4be5b9b39aa05d2bffb4ba8e5ac89572f2 WatchSource:0}: Error finding container af5ff6b4b0d75d4c2b37bd7595e9bf4be5b9b39aa05d2bffb4ba8e5ac89572f2: Status 404 returned error can't find the container with id af5ff6b4b0d75d4c2b37bd7595e9bf4be5b9b39aa05d2bffb4ba8e5ac89572f2 Jul 07 11:08:27.410904 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:27.410875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-g2nwq_eca3c9be-97c5-4abe-b650-7bf5384ffed1/download-server/0.log" Jul 07 11:08:27.671380 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:27.671294 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" event={"ID":"caf0918b-f331-491a-a55f-3ceb52befa20","Type":"ContainerStarted","Data":"4dce1107fef22c2a4acd683b8b0de128a7e7ceb93e15833cb0a936c4deeb86f4"} Jul 07 11:08:27.671380 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:27.671349 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:27.671380 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:27.671365 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" event={"ID":"caf0918b-f331-491a-a55f-3ceb52befa20","Type":"ContainerStarted","Data":"af5ff6b4b0d75d4c2b37bd7595e9bf4be5b9b39aa05d2bffb4ba8e5ac89572f2"} Jul 07 11:08:27.689072 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:27.689017 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" podStartSLOduration=1.689002777 podStartE2EDuration="1.689002777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:08:27.68619289 +0000 UTC m=+2475.146727634" watchObservedRunningTime="2026-07-07 11:08:27.689002777 +0000 UTC m=+2475.149537522" Jul 07 11:08:28.020974 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:28.020879 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-vgx6x_294c4fe3-fc00-4825-b275-ad88fa4199c2/volume-data-source-validator/0.log" Jul 07 11:08:29.331033 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:29.330997 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7ltr2_b127bf4a-b560-4b09-8f9a-21fe84bc3dfe/dns/0.log" Jul 07 11:08:29.356228 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:29.356196 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7ltr2_b127bf4a-b560-4b09-8f9a-21fe84bc3dfe/kube-rbac-proxy/0.log" Jul 07 11:08:29.563440 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:29.563408 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr9zd_601895bc-5e06-4dd8-bdf1-43babbb67529/dns-node-resolver/0.log" Jul 07 11:08:30.159373 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:30.159333 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d8d4559f6-k4fxt_8b42335c-84e9-4349-9ad9-e9ba7fc41e00/registry/0.log" Jul 07 11:08:30.238613 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:30.238580 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vfdgc_55bcc27c-62e9-4172-8855-09ac0c8cb194/node-ca/0.log" Jul 07 11:08:31.260790 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.260760 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4f4t7h_95b07372-b3d8-4bf5-b464-ca2746b7dc85/istio-proxy/0.log" Jul 07 11:08:31.425916 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.425879 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c48467d48-ml746_d8825205-6632-4ee8-b4e7-93a2f856f0eb/kube-auth-proxy/0.log" Jul 07 11:08:31.462865 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.462833 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/0.log" Jul 07 11:08:31.514733 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.514643 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/1.log" Jul 07 11:08:31.574752 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.574721 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fkfld_30b658d7-ce36-48e9-a904-a8cc598deb70/payload-pre-processing/0.log" Jul 07 11:08:31.660017 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:31.659985 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68d68d4cdf-ghwnt_438503da-c32e-49be-9296-4c1beb56f484/router/0.log" Jul 07 11:08:32.422238 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:32.422207 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lkwtd_42d87c1b-6ae0-4156-87ff-bdb1b0c77b76/serve-healthcheck-canary/0.log" Jul 07 11:08:32.952562 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:32.952525 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-tf2f8_ec72510f-f217-42bf-90a6-05669e0f08fb/insights-operator/0.log" Jul 07 11:08:32.953256 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:32.953234 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-tf2f8_ec72510f-f217-42bf-90a6-05669e0f08fb/insights-operator/1.log" Jul 07 11:08:33.067963 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:33.067928 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8qswp_5d34c59e-c9d0-41af-88ea-fd845c1689de/kube-rbac-proxy/0.log" Jul 07 11:08:33.095748 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:33.095712 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8qswp_5d34c59e-c9d0-41af-88ea-fd845c1689de/exporter/0.log" Jul 07 11:08:33.122672 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:33.122645 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8qswp_5d34c59e-c9d0-41af-88ea-fd845c1689de/extractor/0.log" Jul 07 11:08:33.686918 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:33.686887 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rgbf7/perf-node-gather-daemonset-mfcsg" Jul 07 11:08:35.995395 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:35.995356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-swm8t_ff3a74ee-75bb-4ad1-ac14-733562b9197d/manager/0.log" Jul 07 11:08:36.225927 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.225835 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lvrtz_498a8dba-7f57-449d-a0c3-051d6b00e683/manager/0.log" Jul 07 11:08:36.508606 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.508506 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7fdd849877-9kxz9_a6768bfd-1a7e-49a1-853b-4140138e2ade/maas-api/0.log" Jul 07 11:08:36.724306 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.724263 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bcd9f7d5c-lbhrl_182ce1ca-b7ca-4886-bacb-7d69bbd5d37c/manager/0.log" Jul 07 11:08:36.862611 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.862517 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-74927_6c27825c-ada8-4a89-9734-b39cb4622193/manager/0.log" Jul 07 11:08:36.952189 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.952152 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-586fcf4d5b-f94lq_4631b4a6-fb49-43eb-bb12-ce5590455817/manager/0.log" Jul 07 11:08:36.974074 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:36.974049 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-586fcf4d5b-f94lq_4631b4a6-fb49-43eb-bb12-ce5590455817/copy-manifests/0.log" Jul 07 11:08:37.098710 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.098656 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8tjtm_e78edd62-7446-4726-8ba6-7236de296c87/postgres/0.log" Jul 07 11:08:37.222312 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.222285 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4f4t7h_95b07372-b3d8-4bf5-b464-ca2746b7dc85/istio-proxy/0.log" Jul 07 11:08:37.357029 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.357003 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c48467d48-ml746_d8825205-6632-4ee8-b4e7-93a2f856f0eb/kube-auth-proxy/0.log" Jul 07 11:08:37.376605 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.376573 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/0.log" Jul 07 11:08:37.434076 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.434044 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/1.log" Jul 07 11:08:37.465868 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.465834 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fkfld_30b658d7-ce36-48e9-a904-a8cc598deb70/payload-pre-processing/0.log" Jul 07 11:08:37.519512 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:37.519414 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68d68d4cdf-ghwnt_438503da-c32e-49be-9296-4c1beb56f484/router/0.log" Jul 07 11:08:38.562429 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.562352 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5dcl4_43cafda0-7b90-45b1-971f-fc025ae8ac7e/openshift-lws-operator/0.log" Jul 07 11:08:38.745307 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.745279 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4f4t7h_95b07372-b3d8-4bf5-b464-ca2746b7dc85/istio-proxy/0.log" Jul 07 11:08:38.878534 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.878508 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c48467d48-ml746_d8825205-6632-4ee8-b4e7-93a2f856f0eb/kube-auth-proxy/0.log" Jul 07 11:08:38.902624 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.902598 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/0.log" Jul 07 11:08:38.958212 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.958173 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2nhsv_3a4eaadc-8284-4fa1-8c0d-5a159484a916/istio-proxy/1.log" Jul 07 11:08:38.993688 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:38.993653 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-fkfld_30b658d7-ce36-48e9-a904-a8cc598deb70/payload-pre-processing/0.log" Jul 07 11:08:39.047267 ip-10-0-139-204 kubenswrapper[2581]: I0707 11:08:39.047238 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68d68d4cdf-ghwnt_438503da-c32e-49be-9296-4c1beb56f484/router/0.log"