Jul 10 02:20:51.213265 ip-10-0-133-192 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 02:20:51.213275 ip-10-0-133-192 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 02:20:51.213282 ip-10-0-133-192 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 02:20:51.213489 ip-10-0-133-192 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 02:21:01.379600 ip-10-0-133-192 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 02:21:01.379617 ip-10-0-133-192 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot bb5745bcb995403099240d716c9561a3 -- Jul 10 02:22:51.511913 ip-10-0-133-192 systemd[1]: Starting Kubernetes Kubelet... Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.856540 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863561 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863573 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863577 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863581 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863584 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863587 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863589 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:51.878822 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863592 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863595 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863599 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863603 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863606 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863609 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863612 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863615 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863618 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863621 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863624 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863626 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863629 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863632 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863635 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863638 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863641 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863646 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863648 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863651 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:51.886296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863654 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863657 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863672 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863676 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863679 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863682 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863685 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863688 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863690 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863694 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863696 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863699 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863702 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863704 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863707 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863709 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863712 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863715 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863717 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863720 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:51.887171 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863723 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863725 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863728 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863731 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863733 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863736 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863739 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863741 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863744 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863746 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863749 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863752 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863754 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863756 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863759 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863762 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863764 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863767 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863769 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863772 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:51.888073 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863775 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863777 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863780 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863783 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863786 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863788 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863791 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863793 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863796 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863798 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863802 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863805 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863807 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863810 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863813 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863816 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863818 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863821 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863823 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863826 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:51.888949 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863828 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863830 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863833 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863836 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863840 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863844 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863848 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863851 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863854 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863857 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863861 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.863864 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864566 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864572 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864575 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864579 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864582 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864586 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864589 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:51.890049 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864591 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864594 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864597 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864600 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864603 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864606 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864608 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864611 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864614 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864617 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864619 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864622 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864624 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864628 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864631 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864635 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864639 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864642 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864646 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864649 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:51.890937 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864652 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864655 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864657 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864674 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864677 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864680 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864683 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864686 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864689 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864691 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864694 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864697 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864700 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864703 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864706 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864710 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864713 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864716 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864718 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864722 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:51.891817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864724 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864727 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864730 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864733 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864735 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864738 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864740 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864743 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864745 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864748 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864751 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864753 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864756 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864759 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864762 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864764 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864767 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864772 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864774 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:51.892903 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864777 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864780 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864783 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864786 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864788 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864791 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864794 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864797 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864799 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864802 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864805 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864807 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864810 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864813 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864815 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864817 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864820 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864822 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864825 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864827 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:51.893986 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864830 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864832 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864835 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864837 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864840 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864843 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864847 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864850 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864853 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864855 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864860 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864862 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.864865 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865231 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865236 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865238 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865241 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865245 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865248 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:51.894869 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865251 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865253 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865256 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865258 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865261 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865264 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865267 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865269 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865272 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865274 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865277 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865279 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865282 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865284 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865287 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865290 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865292 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865295 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865297 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:51.895941 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865300 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865303 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865305 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865308 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865312 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865315 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865317 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865320 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865323 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865325 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865328 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865330 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865334 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865336 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865339 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865341 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865344 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865347 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865351 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865354 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:51.896813 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865357 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865360 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865363 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865365 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865368 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865370 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865372 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865375 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865377 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865380 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865384 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865387 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865391 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865394 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865397 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865399 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865403 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865406 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865409 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865411 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:51.897690 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865414 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865416 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865419 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865421 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865424 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865427 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865429 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865432 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865435 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865438 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865441 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865443 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865448 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865451 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865455 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865457 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865460 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865463 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865465 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:51.898554 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865468 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865470 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865473 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865475 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865478 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865480 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865483 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865485 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865488 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865492 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865494 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865497 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865502 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865504 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.865507 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865571 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865578 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865584 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865588 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865592 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865596 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 02:22:51.899420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865600 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865604 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865607 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865610 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865613 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865617 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865619 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865622 2560 flags.go:64] FLAG: --cgroup-root="" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865625 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865628 2560 flags.go:64] FLAG: --client-ca-file="" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865631 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865634 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865638 2560 flags.go:64] FLAG: --cluster-domain="" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865640 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865643 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865647 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865650 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865654 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865656 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865674 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865679 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865682 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865685 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865690 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 02:22:51.900285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865693 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865696 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865700 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865703 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865706 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865709 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865712 2560 flags.go:64] FLAG: --enable-server="true" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865715 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865720 2560 flags.go:64] FLAG: --event-burst="100" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865723 2560 flags.go:64] FLAG: --event-qps="50" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865726 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865728 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865731 2560 flags.go:64] FLAG: --eviction-hard="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865735 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865738 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865741 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865744 2560 flags.go:64] FLAG: --eviction-soft="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865746 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865749 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865752 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865755 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865758 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865761 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865764 2560 flags.go:64] FLAG: --feature-gates="" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865767 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 02:22:51.901369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865770 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865773 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865776 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865780 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865783 2560 flags.go:64] FLAG: --help="false" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865786 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-133-192.ec2.internal" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865790 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865793 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865796 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865800 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865803 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865806 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865809 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865812 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865814 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865818 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865821 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865824 2560 flags.go:64] FLAG: --kube-reserved="" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865826 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865829 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865832 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865835 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865838 2560 flags.go:64] FLAG: --lock-file="" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865841 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 02:22:51.902455 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865844 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865847 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865852 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865855 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865858 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865861 2560 flags.go:64] FLAG: --logging-format="text" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865863 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865867 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865870 2560 flags.go:64] FLAG: --manifest-url="" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865872 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865876 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865881 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865885 2560 flags.go:64] FLAG: --max-pods="110" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865888 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865892 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865895 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865898 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865901 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865904 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865907 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865913 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865916 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865919 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 02:22:51.903519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865922 2560 flags.go:64] FLAG: --pod-cidr="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865926 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865931 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865934 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865936 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865939 2560 flags.go:64] FLAG: --port="10250" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865942 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865945 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00b1af5012c4ca5b1" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865948 2560 flags.go:64] FLAG: --qos-reserved="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865951 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865954 2560 flags.go:64] FLAG: --register-node="true" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865957 2560 flags.go:64] FLAG: --register-with-taints="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865960 2560 flags.go:64] FLAG: --registry-burst="10" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865963 2560 flags.go:64] FLAG: --registry-qps="5" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865966 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865969 2560 flags.go:64] FLAG: --reserved-memory="" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865972 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865976 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865979 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865982 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865984 2560 flags.go:64] FLAG: --runonce="false" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865989 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865992 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865996 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.865999 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 02:22:51.904607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866002 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866005 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866008 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866011 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866014 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866017 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866019 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866022 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866025 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866029 2560 flags.go:64] FLAG: --system-cgroups="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866032 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866037 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866039 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866042 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866046 2560 flags.go:64] FLAG: --tls-min-version="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866049 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866052 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866054 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866057 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866060 2560 flags.go:64] FLAG: --v="2" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866063 2560 flags.go:64] FLAG: --version="false" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866067 2560 flags.go:64] FLAG: --vmodule="" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866071 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866074 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866183 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:52.038079 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866187 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:51.931977 ip-10-0-133-192 systemd[1]: Started Kubernetes Kubelet. Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866190 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866192 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866196 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866199 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866204 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866208 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866211 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866214 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866216 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866219 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866222 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866225 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866228 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866231 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866234 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866237 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866240 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866242 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866245 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:52.039511 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866248 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866250 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866253 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866255 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866258 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866260 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866263 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866266 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866268 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866271 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866273 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866276 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866278 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866281 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866283 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866287 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866290 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866294 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866297 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:53.179817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866299 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866302 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866305 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866307 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866310 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866313 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866315 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866318 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866320 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866323 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866325 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866328 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866330 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866333 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866336 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866338 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866341 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866345 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866349 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866351 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:53.180793 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866354 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866356 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866359 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866361 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866364 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866366 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866369 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866371 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866375 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866378 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866382 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866384 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866387 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866389 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866392 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866395 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866397 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866399 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866402 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866405 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:53.181569 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866407 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866409 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866412 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866415 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866418 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866422 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866425 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866428 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866431 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866434 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866437 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866439 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866442 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866444 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866447 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866450 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866452 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866455 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:53.182586 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.866457 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.866958 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.873381 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.873395 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.873413 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.873418 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873471 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873476 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873480 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873483 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873486 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873489 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873492 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873495 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:53.183471 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873498 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873500 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873503 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873506 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873509 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873512 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873514 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873517 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873520 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873523 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873526 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873528 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873532 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873535 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873537 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873540 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873543 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873545 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873548 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873550 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:53.184348 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873553 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873556 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873558 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873561 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873564 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873567 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873569 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873572 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873575 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873577 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873580 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873582 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873585 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873587 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873590 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873592 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873595 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873598 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873601 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873604 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:53.185603 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873607 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873609 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873612 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873614 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873618 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873622 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873625 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873628 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873631 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873634 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873637 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873640 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873643 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873645 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873648 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873651 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873656 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873674 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873679 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:53.186726 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873682 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873685 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873689 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873693 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873696 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873699 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873701 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873704 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873707 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873710 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873713 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873716 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873719 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873722 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873724 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873727 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873729 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873733 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873737 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:53.187785 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873739 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873743 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873746 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873749 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873751 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873754 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873757 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873759 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873762 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873765 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873768 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873771 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873773 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.873779 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873887 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:22:53.188841 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873893 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873896 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873899 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873903 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873906 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873909 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873911 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873914 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873916 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873919 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873922 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873925 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873927 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873930 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873932 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873935 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873937 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873940 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873942 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:22:53.189701 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873946 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873948 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873951 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873954 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873957 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873959 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873962 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873964 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873968 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873970 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873973 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873975 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873978 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873980 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873983 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873985 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873988 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873990 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873993 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873996 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:22:53.190727 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.873998 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874001 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874003 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874006 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874009 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874011 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874014 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874017 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874019 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874022 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874024 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874026 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874030 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874033 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874035 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874038 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874041 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874044 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874046 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:22:53.191602 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874048 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874054 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874057 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874060 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874063 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874066 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874069 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874072 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874074 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874077 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874079 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874082 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874084 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874087 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874089 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874092 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874094 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874097 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874100 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:22:53.192470 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874102 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874105 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874107 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874110 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874112 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874115 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874119 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874122 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874124 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874127 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874129 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874132 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874134 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874137 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874140 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874143 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874145 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874148 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874150 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874153 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:22:53.193548 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:51.874155 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.874160 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.874250 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.876114 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.877125 2560 server.go:1019] "Starting client certificate rotation" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.877224 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.877264 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.881749 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.889160 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.899688 2560 log.go:25] "Validated CRI v1 runtime API" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.901723 2560 log.go:25] "Validated CRI v1 image API" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.903078 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 02:22:53.194427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.910499 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 02:22:53.195072 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.910745 2560 fs.go:135] Filesystem UUIDs: map[1eb06bcc-1c9f-4753-99a2-42342a36ced8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bd2de77f-42e3-43d6-88a7-10b5884f3f7c:/dev/nvme0n1p4] Jul 10 02:22:53.195072 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.910770 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.915739 2560 manager.go:216] Machine: {Timestamp:2026-07-10 02:22:51.914838888 +0000 UTC m=+0.300233732 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099668 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e780a51d83befe591830f62465d2e SystemUUID:ec2e780a-51d8-3bef-e591-830f62465d2e BootID:bb5745bc-b995-4030-9924-0d716c9561a3 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:af:cf:ca:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:af:cf:ca:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:c9:60:d9:7c:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.915833 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.915937 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.916780 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.916804 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-192.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.917422 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.917431 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.917438 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.917945 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.918941 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922476 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922607 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922634 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922650 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922677 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922687 2560 kubelet.go:398] "Adding apiserver pod source" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.922698 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 02:22:53.195503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.923028 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.923043 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.924758 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926329 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926357 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926517 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926530 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926535 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926540 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926549 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.926557 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927056 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927062 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927069 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927074 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927086 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927095 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927400 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7qhht" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927809 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.927818 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.931120 2560 server.go:1268] "Started kubelet" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.931581 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.931719 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.931792 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.933459 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.933528 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-192.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.933557 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.933843 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.933898 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7qhht" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.936903 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.937192 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.938045 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.938063 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.938152 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.938374 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.938710 2560 factory.go:55] Registering systemd factory Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.938722 2560 factory.go:223] Registration of the systemd container factory successfully Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.953682 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.953694 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.953876 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954227 2560 factory.go:153] Registering CRI-O factory Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954242 2560 factory.go:223] Registration of the crio container factory successfully Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954315 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954338 2560 factory.go:103] Registering Raw factory Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954352 2560 manager.go:1196] Started watching for new ooms in manager Jul 10 02:22:53.196375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.954871 2560 manager.go:349] Starting recovery of all containers Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.953875 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-192.ec2.internal.18c0cc619584714a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-192.ec2.internal,UID:ip-10-0-133-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-192.ec2.internal,},FirstTimestamp:2026-07-10 02:22:51.93109537 +0000 UTC m=+0.316490080,LastTimestamp:2026-07-10 02:22:51.93109537 +0000 UTC m=+0.316490080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-192.ec2.internal,}" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.957471 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.957585 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:51.960310 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-192.ec2.internal\" not found" node="ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.965512 2560 manager.go:354] Recovery completed Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.969550 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979356 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979388 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979405 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979926 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979935 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979951 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.979938 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.980635 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.980673 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.980693 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.982155 2560 policy_none.go:49] "None policy: Start" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.982169 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.982179 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:51.984165 2560 policy_none.go:47] "Start" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009179 2560 manager.go:337] "Starting Device Plugin manager" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:52.009321 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009331 2560 server.go:87] "Starting device plugin registration server" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009597 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009611 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009702 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009788 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:52.009797 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:52.010255 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 02:22:53.200836 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:52.010306 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.216810 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.217891 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.217922 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.217943 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.217972 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.218321 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.219737 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.219774 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.219790 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.219822 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.220533 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.231754 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.231772 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-192.ec2.internal\": node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.247765 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.318425 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal"] Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.318506 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.319432 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.319462 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.319475 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.321873 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.321991 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322019 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322574 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322578 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322594 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322602 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322608 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.322613 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.324853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.324884 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.325457 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.325473 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.325482 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.345764 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-192.ec2.internal\" not found" node="ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.347812 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.349557 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-192.ec2.internal\" not found" node="ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.433873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.363562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.438125 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.363592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95c7e23d0cc9fb766a175e924a681da-config\") pod \"kube-apiserver-proxy-ip-10-0-133-192.ec2.internal\" (UID: \"d95c7e23d0cc9fb766a175e924a681da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.438125 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.363611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.447993 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464542 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95c7e23d0cc9fb766a175e924a681da-config\") pod \"kube-apiserver-proxy-ip-10-0-133-192.ec2.internal\" (UID: \"d95c7e23d0cc9fb766a175e924a681da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95c7e23d0cc9fb766a175e924a681da-config\") pod \"kube-apiserver-proxy-ip-10-0-133-192.ec2.internal\" (UID: \"d95c7e23d0cc9fb766a175e924a681da\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464590 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.464605 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c1b6fa54a62197cf3a68020c079a41f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal\" (UID: \"1c1b6fa54a62197cf3a68020c079a41f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.549055 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.647397 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.651493 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.649503 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.652137 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.651647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:53.900487 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.750051 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:53.900487 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.850496 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:54.028488 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:53.950941 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:54.028488 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.985222 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 02:22:54.028488 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.985347 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 02:22:54.028488 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:53.985347 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 02:22:54.153950 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.049550 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:17:53 +0000 UTC" deadline="2028-03-31 01:55:10.181304814 +0000 UTC" Jul 10 02:22:54.153950 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.049569 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15119h32m16.131738127s" Jul 10 02:22:54.153950 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:54.051672 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-192.ec2.internal\" not found" Jul 10 02:22:54.153950 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.104922 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:54.153950 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.147125 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" Jul 10 02:22:54.157915 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.157156 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 02:22:54.159694 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.158899 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" Jul 10 02:22:54.270683 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.171591 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 02:22:54.391210 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.334931 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:54.825110 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:54.788292 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1b6fa54a62197cf3a68020c079a41f.slice/crio-d71f22f30678dd696f4c3056d5624564dd45d118b6ad3c3c4ff8e9d76ef4fb1f WatchSource:0}: Error finding container d71f22f30678dd696f4c3056d5624564dd45d118b6ad3c3c4ff8e9d76ef4fb1f: Status 404 returned error can't find the container with id d71f22f30678dd696f4c3056d5624564dd45d118b6ad3c3c4ff8e9d76ef4fb1f Jul 10 02:22:54.825110 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:54.788848 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95c7e23d0cc9fb766a175e924a681da.slice/crio-c37bf3c85ca8daf5172a2703524db73f5c1c478f6fb9a91c07ddd35514b47fed WatchSource:0}: Error finding container c37bf3c85ca8daf5172a2703524db73f5c1c478f6fb9a91c07ddd35514b47fed: Status 404 returned error can't find the container with id c37bf3c85ca8daf5172a2703524db73f5c1c478f6fb9a91c07ddd35514b47fed Jul 10 02:22:54.825110 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:54.798154 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.031782 2560 apiserver.go:52] "Watching apiserver" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.039409 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.040919 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-htnk4","kube-system/global-pull-secret-syncer-gqsgv","kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl","openshift-dns/node-resolver-7lpds","openshift-image-registry/node-ca-mszkn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal","openshift-network-diagnostics/network-check-target-d4b7g","kube-system/konnectivity-agent-d7787","openshift-cluster-node-tuning-operator/tuned-5fmkb","openshift-multus/multus-82xkg","openshift-multus/multus-additional-cni-plugins-v8sj7","openshift-multus/network-metrics-daemon-j6wg4","openshift-network-operator/iptables-alerter-c4x5k"] Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.044821 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.046960 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.047022 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.047435 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.047483 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.047544 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6pxzv\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.048248 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.049063 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.051315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.051368 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.051400 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.051416 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlpg9\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.051719 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.053404 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.053457 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.053655 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.053724 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.053825 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2pl9h\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.055369 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.055580 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.056024 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.056099 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-487wq\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.057546 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.057633 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.057709 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.057814 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dhfx6\"" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.057946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.078034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.059832 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.059892 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.060081 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.060156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t2lcm\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.060207 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.060225 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.061905 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.061957 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.064061 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.066319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.066498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.067200 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.067015 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.068842 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.069682 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.069706 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-conf\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-etc-tuned\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070162 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-iptables-alerter-script\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070168 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070178 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cw7z5\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070185 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2lkjl\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070244 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czsnk\" (UniqueName: \"kubernetes.io/projected/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-kube-api-access-czsnk\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070317 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-var-lib-kubelet\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070333 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 02:22:55.080643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-host\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-kubernetes\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-sys\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070413 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-tmp\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070439 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2xg8\" (UniqueName: \"kubernetes.io/projected/179493af-371a-414e-9602-5de5cae4d7a8-kube-api-access-q2xg8\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysconfig\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-modprobe-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-systemd\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-run\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-lib-modules\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpw7j\" (UniqueName: \"kubernetes.io/projected/6d7e586d-7299-4425-b801-8abc86a7088d-kube-api-access-wpw7j\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070635 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.070649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-host-slash\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.071652 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.071853 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 02:22:55.081961 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.072196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-984g8\"" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.092224 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c8p2w" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.101682 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c8p2w" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.148479 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-modprobe-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171466 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-systemd\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171491 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab9baa9e-314f-4d93-aba3-0bc72da716c5-tmp-dir\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171511 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-netns\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171546 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pvj\" (UniqueName: \"kubernetes.io/projected/38976ef2-7c34-4df8-9252-337e2e01b774-kube-api-access-v8pvj\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-modprobe-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-iptables-alerter-script\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-systemd\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cnibin\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171699 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-socket-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-log-socket\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpkz6\" (UniqueName: \"kubernetes.io/projected/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-kube-api-access-wpkz6\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171781 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-host\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.214280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-kubelet\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-script-lib\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171865 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wj7\" (UniqueName: \"kubernetes.io/projected/6d4ef782-8911-480a-90dd-63d7de6699f3-kube-api-access-74wj7\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-multus\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171919 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-host\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.171925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-kubelet-config\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172120 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-env-overrides\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38976ef2-7c34-4df8-9252-337e2e01b774-ovn-node-metrics-cert\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-multus-daemon-config\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172216 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-iptables-alerter-script\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172244 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-dbus\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172354 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-systemd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-device-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.220536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-ovn\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-kubernetes\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172465 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-kubernetes\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-os-release\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5zb2\" (UniqueName: \"kubernetes.io/projected/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kube-api-access-k5zb2\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-config\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aba6bc18-cab9-41dd-a432-32bfe37df3df-agent-certs\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172580 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-cni-binary-copy\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-run\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-host-slash\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-cnibin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-d\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-sys-fs\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172737 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-host-slash\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172738 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-run\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.221507 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172741 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-systemd-units\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-slash\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-etc-tuned\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-socket-dir-parent\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-k8s-cni-cncf-io\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-hostroot\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-etc-kubernetes\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.172989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt9k2\" (UniqueName: \"kubernetes.io/projected/6b50a459-0940-404b-b449-080620155638-kube-api-access-mt9k2\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-registration-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-os-release\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d4ef782-8911-480a-90dd-63d7de6699f3-serviceca\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173075 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-sys\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab9baa9e-314f-4d93-aba3-0bc72da716c5-hosts-file\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aba6bc18-cab9-41dd-a432-32bfe37df3df-konnectivity-ca\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173182 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-bin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173189 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-sys\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.222509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-etc-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysconfig\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-conf-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173371 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysconfig\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-lib-modules\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpw7j\" (UniqueName: \"kubernetes.io/projected/6d7e586d-7299-4425-b801-8abc86a7088d-kube-api-access-wpw7j\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173570 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-lib-modules\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173681 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-kubelet\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.173808 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.173829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-conf\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.174029 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:55.674007821 +0000 UTC m=+2.950797255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174063 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-etc-sysctl-conf\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174205 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czsnk\" (UniqueName: \"kubernetes.io/projected/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-kube-api-access-czsnk\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174257 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hftp\" (UniqueName: \"kubernetes.io/projected/ab9baa9e-314f-4d93-aba3-0bc72da716c5-kube-api-access-2hftp\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.297004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174508 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d4ef782-8911-480a-90dd-63d7de6699f3-host\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-system-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174588 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-multus-certs\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-var-lib-kubelet\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-tmp\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2xg8\" (UniqueName: \"kubernetes.io/projected/179493af-371a-414e-9602-5de5cae4d7a8-kube-api-access-q2xg8\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-var-lib-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d7e586d-7299-4425-b801-8abc86a7088d-var-lib-kubelet\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-netns\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-node-log\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.298018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-bin\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.174974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-netd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.176883 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-etc-tuned\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.176989 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d7e586d-7299-4425-b801-8abc86a7088d-tmp\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.184987 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpw7j\" (UniqueName: \"kubernetes.io/projected/6d7e586d-7299-4425-b801-8abc86a7088d-kube-api-access-wpw7j\") pod \"tuned-5fmkb\" (UID: \"6d7e586d-7299-4425-b801-8abc86a7088d\") " pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.193915 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czsnk\" (UniqueName: \"kubernetes.io/projected/3dd8b7cb-f267-40e5-8b83-cba4df3e00b4-kube-api-access-czsnk\") pod \"iptables-alerter-c4x5k\" (UID: \"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4\") " pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.194355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2xg8\" (UniqueName: \"kubernetes.io/projected/179493af-371a-414e-9602-5de5cae4d7a8-kube-api-access-q2xg8\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.221297 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.227150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" event={"ID":"d95c7e23d0cc9fb766a175e924a681da","Type":"ContainerStarted","Data":"c37bf3c85ca8daf5172a2703524db73f5c1c478f6fb9a91c07ddd35514b47fed"} Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.228553 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" event={"ID":"1c1b6fa54a62197cf3a68020c079a41f","Type":"ContainerStarted","Data":"d71f22f30678dd696f4c3056d5624564dd45d118b6ad3c3c4ff8e9d76ef4fb1f"} Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275191 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-os-release\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5zb2\" (UniqueName: \"kubernetes.io/projected/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kube-api-access-k5zb2\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-config\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aba6bc18-cab9-41dd-a432-32bfe37df3df-agent-certs\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-cni-binary-copy\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-cnibin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-sys-fs\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.299148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-systemd-units\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275434 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-slash\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-socket-dir-parent\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-k8s-cni-cncf-io\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-hostroot\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-etc-kubernetes\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt9k2\" (UniqueName: \"kubernetes.io/projected/6b50a459-0940-404b-b449-080620155638-kube-api-access-mt9k2\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-registration-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-os-release\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d4ef782-8911-480a-90dd-63d7de6699f3-serviceca\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab9baa9e-314f-4d93-aba3-0bc72da716c5-hosts-file\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aba6bc18-cab9-41dd-a432-32bfe37df3df-konnectivity-ca\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-bin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-etc-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-conf-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-kubelet\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.299994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hftp\" (UniqueName: \"kubernetes.io/projected/ab9baa9e-314f-4d93-aba3-0bc72da716c5-kube-api-access-2hftp\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d4ef782-8911-480a-90dd-63d7de6699f3-host\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.275903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-etc-kubernetes\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-bin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-os-release\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-kubelet\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276714 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-etc-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276746 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-conf-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-registration-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.276945 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-os-release\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ab9baa9e-314f-4d93-aba3-0bc72da716c5-hosts-file\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277176 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-sys-fs\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-config\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-systemd-units\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.301094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277387 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-slash\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d4ef782-8911-480a-90dd-63d7de6699f3-serviceca\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277441 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-socket-dir-parent\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277486 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-k8s-cni-cncf-io\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277526 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-hostroot\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277581 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d4ef782-8911-480a-90dd-63d7de6699f3-host\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.277631 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-cnibin\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.280493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aba6bc18-cab9-41dd-a432-32bfe37df3df-agent-certs\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.280868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.286873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt9k2\" (UniqueName: \"kubernetes.io/projected/6b50a459-0940-404b-b449-080620155638-kube-api-access-mt9k2\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.287963 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-system-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.287992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-multus-certs\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aba6bc18-cab9-41dd-a432-32bfe37df3df-konnectivity-ca\") pod \"konnectivity-agent-d7787\" (UID: \"aba6bc18-cab9-41dd-a432-32bfe37df3df\") " pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-cni-binary-copy\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288049 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-system-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288071 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-multus-certs\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303055 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-var-lib-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-netns\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-node-log\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-bin\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-multus-cni-dir\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-netd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab9baa9e-314f-4d93-aba3-0bc72da716c5-tmp-dir\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-netns\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pvj\" (UniqueName: \"kubernetes.io/projected/38976ef2-7c34-4df8-9252-337e2e01b774-kube-api-access-v8pvj\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288310 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-netd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cnibin\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cnibin\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-socket-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.303831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-log-socket\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpkz6\" (UniqueName: \"kubernetes.io/projected/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-kube-api-access-wpkz6\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-kubelet\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-script-lib\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74wj7\" (UniqueName: \"kubernetes.io/projected/6d4ef782-8911-480a-90dd-63d7de6699f3-kube-api-access-74wj7\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288512 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab9baa9e-314f-4d93-aba3-0bc72da716c5-tmp-dir\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-var-lib-openvswitch\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-netns\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288563 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-run-netns\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288613 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-run-ovn-kubernetes\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288642 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-node-log\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.288693 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-log-socket\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.288739 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:55.788726395 +0000 UTC m=+3.065515819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:55.304528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288739 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-socket-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288932 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-kubelet\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-multus\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289062 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-ovnkube-script-lib\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.288867 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-host-cni-bin\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289406 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b50a459-0940-404b-b449-080620155638-host-var-lib-cni-multus\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-kubelet-config\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289511 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-env-overrides\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38976ef2-7c34-4df8-9252-337e2e01b774-ovn-node-metrics-cert\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-system-cni-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289600 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-kubelet-config\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-multus-daemon-config\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-dbus\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-systemd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-device-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289793 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-ovn\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289852 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-ovn\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.289980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hftp\" (UniqueName: \"kubernetes.io/projected/ab9baa9e-314f-4d93-aba3-0bc72da716c5-kube-api-access-2hftp\") pod \"node-resolver-7lpds\" (UID: \"ab9baa9e-314f-4d93-aba3-0bc72da716c5\") " pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b50a459-0940-404b-b449-080620155638-multus-daemon-config\") pod \"multus-82xkg\" (UID: \"6b50a459-0940-404b-b449-080620155638\") " pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38976ef2-7c34-4df8-9252-337e2e01b774-run-systemd\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290193 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-dbus\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a04013e6-a20f-4770-b890-1c3e32dbe1ad-device-dir\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.290435 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38976ef2-7c34-4df8-9252-337e2e01b774-env-overrides\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.292179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5zb2\" (UniqueName: \"kubernetes.io/projected/a04013e6-a20f-4770-b890-1c3e32dbe1ad-kube-api-access-k5zb2\") pod \"aws-ebs-csi-driver-node-zsztl\" (UID: \"a04013e6-a20f-4770-b890-1c3e32dbe1ad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.294126 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38976ef2-7c34-4df8-9252-337e2e01b774-ovn-node-metrics-cert\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.297490 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.297511 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.297524 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:55.305899 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.297580 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:22:55.797563451 +0000 UTC m=+3.074352880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:55.307343 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.307323 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpkz6\" (UniqueName: \"kubernetes.io/projected/34b111d8-1f9a-41a2-baf7-2ad06a1a04a9-kube-api-access-wpkz6\") pod \"multus-additional-cni-plugins-v8sj7\" (UID: \"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9\") " pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.308765 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.308739 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wj7\" (UniqueName: \"kubernetes.io/projected/6d4ef782-8911-480a-90dd-63d7de6699f3-kube-api-access-74wj7\") pod \"node-ca-mszkn\" (UID: \"6d4ef782-8911-480a-90dd-63d7de6699f3\") " pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.309129 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.309103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pvj\" (UniqueName: \"kubernetes.io/projected/38976ef2-7c34-4df8-9252-337e2e01b774-kube-api-access-v8pvj\") pod \"ovnkube-node-htnk4\" (UID: \"38976ef2-7c34-4df8-9252-337e2e01b774\") " pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.353941 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.353877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" Jul 10 02:22:55.364534 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.364511 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c4x5k" Jul 10 02:22:55.364916 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.364894 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7e586d_7299_4425_b801_8abc86a7088d.slice/crio-c487650271844dec9d13191f789d05177cf58ffe7298475e43af096ef75a4c47 WatchSource:0}: Error finding container c487650271844dec9d13191f789d05177cf58ffe7298475e43af096ef75a4c47: Status 404 returned error can't find the container with id c487650271844dec9d13191f789d05177cf58ffe7298475e43af096ef75a4c47 Jul 10 02:22:55.374637 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.374479 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" Jul 10 02:22:55.374788 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.374770 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:22:55.376537 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.376509 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd8b7cb_f267_40e5_8b83_cba4df3e00b4.slice/crio-e852b7dc5a9d82e7ae9f4dc651300047805f1880bbece7229fbd4392aafc9498 WatchSource:0}: Error finding container e852b7dc5a9d82e7ae9f4dc651300047805f1880bbece7229fbd4392aafc9498: Status 404 returned error can't find the container with id e852b7dc5a9d82e7ae9f4dc651300047805f1880bbece7229fbd4392aafc9498 Jul 10 02:22:55.377982 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.377961 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7lpds" Jul 10 02:22:55.383563 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.383540 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mszkn" Jul 10 02:22:55.386370 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.386249 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04013e6_a20f_4770_b890_1c3e32dbe1ad.slice/crio-4207f402134a1cb8e22dd0e8d0d63b0af9ae0368dbc92d04c7bf1629a9e281e3 WatchSource:0}: Error finding container 4207f402134a1cb8e22dd0e8d0d63b0af9ae0368dbc92d04c7bf1629a9e281e3: Status 404 returned error can't find the container with id 4207f402134a1cb8e22dd0e8d0d63b0af9ae0368dbc92d04c7bf1629a9e281e3 Jul 10 02:22:55.389313 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.389299 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:22:55.393563 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.393543 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9baa9e_314f_4d93_aba3_0bc72da716c5.slice/crio-33e5fc5632f11eec74c70f37f2c2c12f668521f593edfc301ea3f49b1e74560a WatchSource:0}: Error finding container 33e5fc5632f11eec74c70f37f2c2c12f668521f593edfc301ea3f49b1e74560a: Status 404 returned error can't find the container with id 33e5fc5632f11eec74c70f37f2c2c12f668521f593edfc301ea3f49b1e74560a Jul 10 02:22:55.394762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.394741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:22:55.395360 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.395326 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4ef782_8911_480a_90dd_63d7de6699f3.slice/crio-a2d44fa93798ffd5eac06585952984c8ab266818b014095b8535b3eeed7cb4ab WatchSource:0}: Error finding container a2d44fa93798ffd5eac06585952984c8ab266818b014095b8535b3eeed7cb4ab: Status 404 returned error can't find the container with id a2d44fa93798ffd5eac06585952984c8ab266818b014095b8535b3eeed7cb4ab Jul 10 02:22:55.400226 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.399504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-82xkg" Jul 10 02:22:55.404115 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.403966 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" Jul 10 02:22:55.408167 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.408145 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba6bc18_cab9_41dd_a432_32bfe37df3df.slice/crio-34847386334d8cf2dbfd960717521efc234762a30c1c8f92f324314ca41ec2d4 WatchSource:0}: Error finding container 34847386334d8cf2dbfd960717521efc234762a30c1c8f92f324314ca41ec2d4: Status 404 returned error can't find the container with id 34847386334d8cf2dbfd960717521efc234762a30c1c8f92f324314ca41ec2d4 Jul 10 02:22:55.413906 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.413886 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b50a459_0940_404b_b449_080620155638.slice/crio-6ce0518cb264804c672eddf288a5585dd32d9de894a0ef0fccba76a327b51251 WatchSource:0}: Error finding container 6ce0518cb264804c672eddf288a5585dd32d9de894a0ef0fccba76a327b51251: Status 404 returned error can't find the container with id 6ce0518cb264804c672eddf288a5585dd32d9de894a0ef0fccba76a327b51251 Jul 10 02:22:55.415221 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.415202 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38976ef2_7c34_4df8_9252_337e2e01b774.slice/crio-daf6cd76b733e6f0a2d61eceff2879806f36d121eb74794b09c6d7f8280539f7 WatchSource:0}: Error finding container daf6cd76b733e6f0a2d61eceff2879806f36d121eb74794b09c6d7f8280539f7: Status 404 returned error can't find the container with id daf6cd76b733e6f0a2d61eceff2879806f36d121eb74794b09c6d7f8280539f7 Jul 10 02:22:55.419296 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:22:55.419262 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b111d8_1f9a_41a2_baf7_2ad06a1a04a9.slice/crio-5097f0a50d2ca8cb9cd90a0f596aea4f364ff81c502a64e229c529645551afbf WatchSource:0}: Error finding container 5097f0a50d2ca8cb9cd90a0f596aea4f364ff81c502a64e229c529645551afbf: Status 404 returned error can't find the container with id 5097f0a50d2ca8cb9cd90a0f596aea4f364ff81c502a64e229c529645551afbf Jul 10 02:22:55.692555 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.692492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:55.692687 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.692623 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:55.692687 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.692675 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:56.692658894 +0000 UTC m=+3.969448320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:55.793006 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.792974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:55.793155 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.793111 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:55.793214 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.793175 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:56.7931549 +0000 UTC m=+4.069944327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:55.893900 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:55.893871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:55.894067 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.894034 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:22:55.894067 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.894050 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:22:55.894067 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.894059 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:55.894205 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:55.894119 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:22:56.894101351 +0000 UTC m=+4.170890777 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:56.104653 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.104547 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:17:55 +0000 UTC" deadline="2028-02-07 16:20:20.718099998 +0000 UTC" Jul 10 02:22:56.104653 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.104604 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13861h57m24.613500556s" Jul 10 02:22:56.232117 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.232071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lpds" event={"ID":"ab9baa9e-314f-4d93-aba3-0bc72da716c5","Type":"ContainerStarted","Data":"33e5fc5632f11eec74c70f37f2c2c12f668521f593edfc301ea3f49b1e74560a"} Jul 10 02:22:56.233924 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.233894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" event={"ID":"a04013e6-a20f-4770-b890-1c3e32dbe1ad","Type":"ContainerStarted","Data":"4207f402134a1cb8e22dd0e8d0d63b0af9ae0368dbc92d04c7bf1629a9e281e3"} Jul 10 02:22:56.235504 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.235476 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" event={"ID":"6d7e586d-7299-4425-b801-8abc86a7088d","Type":"ContainerStarted","Data":"c487650271844dec9d13191f789d05177cf58ffe7298475e43af096ef75a4c47"} Jul 10 02:22:56.236940 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.236917 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-82xkg" event={"ID":"6b50a459-0940-404b-b449-080620155638","Type":"ContainerStarted","Data":"6ce0518cb264804c672eddf288a5585dd32d9de894a0ef0fccba76a327b51251"} Jul 10 02:22:56.238928 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.238904 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d7787" event={"ID":"aba6bc18-cab9-41dd-a432-32bfe37df3df","Type":"ContainerStarted","Data":"34847386334d8cf2dbfd960717521efc234762a30c1c8f92f324314ca41ec2d4"} Jul 10 02:22:56.240634 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.240610 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mszkn" event={"ID":"6d4ef782-8911-480a-90dd-63d7de6699f3","Type":"ContainerStarted","Data":"a2d44fa93798ffd5eac06585952984c8ab266818b014095b8535b3eeed7cb4ab"} Jul 10 02:22:56.242367 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.242335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c4x5k" event={"ID":"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4","Type":"ContainerStarted","Data":"e852b7dc5a9d82e7ae9f4dc651300047805f1880bbece7229fbd4392aafc9498"} Jul 10 02:22:56.243696 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.243658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerStarted","Data":"5097f0a50d2ca8cb9cd90a0f596aea4f364ff81c502a64e229c529645551afbf"} Jul 10 02:22:56.245034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.245012 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"daf6cd76b733e6f0a2d61eceff2879806f36d121eb74794b09c6d7f8280539f7"} Jul 10 02:22:56.701667 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.701628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:56.701856 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.701788 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:56.701856 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.701852 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:58.701834143 +0000 UTC m=+5.978623570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:56.802525 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.801960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:56.802525 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.802112 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:56.802525 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.802173 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:22:58.802153421 +0000 UTC m=+6.078942865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:56.903126 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:56.903088 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:56.903332 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.903285 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:22:56.903332 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.903304 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:22:56.903332 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.903317 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:56.903527 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:56.903377 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:22:58.903357938 +0000 UTC m=+6.180147399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:57.105080 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:57.105017 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:17:55 +0000 UTC" deadline="2028-02-28 15:58:20.851571194 +0000 UTC" Jul 10 02:22:57.105080 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:57.105057 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14365h35m23.746519263s" Jul 10 02:22:57.219453 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:57.219399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:57.219624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:57.219397 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:57.219624 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:57.219522 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:22:57.219624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:57.219538 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:57.219807 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:57.219629 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:22:57.219807 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:57.219719 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:22:58.266139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.265373 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" event={"ID":"d95c7e23d0cc9fb766a175e924a681da","Type":"ContainerStarted","Data":"b95475974b622ac52d6f406efcb5a7324735e81f7505a047191cf2385e132724"} Jul 10 02:22:58.274908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.274506 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" event={"ID":"1c1b6fa54a62197cf3a68020c079a41f","Type":"ContainerStarted","Data":"267bfb3faaf2a3773168df42c7c05046e5c9d7a464474069552fb81a6cbc87c4"} Jul 10 02:22:58.283022 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.282598 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-192.ec2.internal" podStartSLOduration=4.282580597 podStartE2EDuration="4.282580597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:22:58.282407616 +0000 UTC m=+5.559197064" watchObservedRunningTime="2026-07-10 02:22:58.282580597 +0000 UTC m=+5.559370045" Jul 10 02:22:58.719446 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.718811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:58.719446 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.719004 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:58.719446 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.719068 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:02.719048952 +0000 UTC m=+9.995838381 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:22:58.819573 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.819543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:58.819752 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.819734 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:58.819829 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.819796 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:02.819775677 +0000 UTC m=+10.096565120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:22:58.920778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:58.919918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:58.920778 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.920094 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:22:58.920778 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.920113 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:22:58.920778 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.920126 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:58.920778 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:58.920179 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:23:02.920161054 +0000 UTC m=+10.196950485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:22:59.220038 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:59.219363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:22:59.220038 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:59.219489 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:22:59.220038 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:59.219846 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:22:59.220038 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:59.219932 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:22:59.220604 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:22:59.220455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:22:59.220604 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:22:59.220563 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:01.219287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:01.219023 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:01.219287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:01.219042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:01.219287 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:01.219169 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:01.219287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:01.219174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:01.219851 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:01.219283 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:01.219851 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:01.219367 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:02.750377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:02.750198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:02.750828 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.750404 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:02.750828 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.750478 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:10.750456659 +0000 UTC m=+18.027246097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:02.851596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:02.851556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:02.851766 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.851732 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:02.851823 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.851793 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:10.851774276 +0000 UTC m=+18.128563701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:02.952402 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:02.952367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:02.952564 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.952554 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:23:02.952619 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.952573 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:23:02.952619 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.952586 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:02.952711 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:02.952643 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:23:10.952624638 +0000 UTC m=+18.229414082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:03.219665 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:03.219586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:03.219823 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:03.219692 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:03.219823 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:03.219764 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:03.219950 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:03.219858 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:03.219950 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:03.219896 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:03.220180 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:03.219965 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:04.289363 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:04.289308 2560 generic.go:356] "Generic (PLEG): container finished" podID="1c1b6fa54a62197cf3a68020c079a41f" containerID="267bfb3faaf2a3773168df42c7c05046e5c9d7a464474069552fb81a6cbc87c4" exitCode=0 Jul 10 02:23:04.289831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:04.289375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" event={"ID":"1c1b6fa54a62197cf3a68020c079a41f","Type":"ContainerDied","Data":"267bfb3faaf2a3773168df42c7c05046e5c9d7a464474069552fb81a6cbc87c4"} Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:05.222145 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:05.222164 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:05.222284 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:05.222313 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:05.222448 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:05.222599 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:05.222538 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:07.218444 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:07.218412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:07.218872 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:07.218412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:07.218872 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:07.218532 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:07.218872 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:07.218644 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:07.218872 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:07.218421 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:07.218872 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:07.218768 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:09.221584 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:09.221555 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:09.221994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:09.221685 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:09.221994 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:09.221686 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:09.221994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:09.221696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:09.221994 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:09.221782 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:09.221994 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:09.221868 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:10.811081 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:10.811045 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:10.811611 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:10.811214 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:10.811611 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:10.811304 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:26.81128218 +0000 UTC m=+34.088071610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:10.912324 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:10.912284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:10.912483 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:10.912432 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:10.912562 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:10.912493 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:26.912475263 +0000 UTC m=+34.189264693 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:11.013498 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:11.013466 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:11.013680 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.013615 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:23:11.013680 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.013633 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:23:11.013680 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.013643 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:11.013821 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.013700 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:23:27.013682484 +0000 UTC m=+34.290471909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:11.218559 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:11.218475 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:11.218712 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:11.218492 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:11.218712 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.218605 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:11.218813 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.218706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:11.218813 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:11.218742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:11.218813 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:11.218803 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:13.219119 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:13.219083 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:13.219578 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:13.219182 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:13.219578 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:13.219304 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:13.219578 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:13.219410 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:13.219578 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:13.219442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:13.219578 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:13.219525 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:14.311124 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.310898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" event={"ID":"1c1b6fa54a62197cf3a68020c079a41f","Type":"ContainerStarted","Data":"523a97a196728f8d2367fa31d8e9221edee563edd16e87448b1a0abad32f1102"} Jul 10 02:23:14.313585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.313219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-82xkg" event={"ID":"6b50a459-0940-404b-b449-080620155638","Type":"ContainerStarted","Data":"a42ab549947c880ed551b0bf35901f1540aaf11fc187f750e77908404f3d6097"} Jul 10 02:23:14.316195 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.315777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d7787" event={"ID":"aba6bc18-cab9-41dd-a432-32bfe37df3df","Type":"ContainerStarted","Data":"36f7606b5565ebfd9905b587872a9800973102b7b54485d637aad3cad8cbaee2"} Jul 10 02:23:14.320458 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.320148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"edc80b9c7111f39d6037c5635a7c0cb3a19bfa29e5f876328025f9bbef525bdf"} Jul 10 02:23:14.322583 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.322288 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" event={"ID":"6d7e586d-7299-4425-b801-8abc86a7088d","Type":"ContainerStarted","Data":"5fd1987dba69d389ed7273179bc43747f8508312becf0fac159e07787ad15dbf"} Jul 10 02:23:14.350874 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.350795 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-192.ec2.internal" podStartSLOduration=20.350773061 podStartE2EDuration="20.350773061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:14.327887577 +0000 UTC m=+21.604677023" watchObservedRunningTime="2026-07-10 02:23:14.350773061 +0000 UTC m=+21.627562606" Jul 10 02:23:14.350993 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.350957 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d7787" podStartSLOduration=10.728598275 podStartE2EDuration="21.35094789s" totalImagesPullingTime="10.622349615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:14.350245051 +0000 UTC m=+21.627034496" watchObservedRunningTime="2026-07-10 02:23:14.35094789 +0000 UTC m=+21.627737337" Jul 10 02:23:14.369813 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.369772 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-82xkg" podStartSLOduration=2.664026973 podStartE2EDuration="21.369755376s" totalImagesPullingTime="18.705728403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:14.368789297 +0000 UTC m=+21.645578743" watchObservedRunningTime="2026-07-10 02:23:14.369755376 +0000 UTC m=+21.646544822" Jul 10 02:23:14.386477 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:14.386430 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5fmkb" podStartSLOduration=2.6657720339999997 podStartE2EDuration="21.386412018s" totalImagesPullingTime="18.720639984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:14.386055266 +0000 UTC m=+21.662844713" watchObservedRunningTime="2026-07-10 02:23:14.386412018 +0000 UTC m=+21.663201464" Jul 10 02:23:15.200681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.200653 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:23:15.201259 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.201242 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:23:15.220764 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.220748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:15.220854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.220827 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:15.220854 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:15.220837 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:15.220932 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:15.220895 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:15.220932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.220918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:15.221010 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:15.220991 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:15.324867 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.324843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mszkn" event={"ID":"6d4ef782-8911-480a-90dd-63d7de6699f3","Type":"ContainerStarted","Data":"e1a2b9c2aba461ec4c2d0afcb1784c9e8e85dcec1640bce8241d4b2f5894c60f"} Jul 10 02:23:15.326354 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.326333 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="5c59e65a432bcb94fa496d1aaa05f81f6cf296fa0bfefcdea45f93b94c746dfe" exitCode=0 Jul 10 02:23:15.326434 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.326410 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"5c59e65a432bcb94fa496d1aaa05f81f6cf296fa0bfefcdea45f93b94c746dfe"} Jul 10 02:23:15.329043 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329027 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:23:15.329400 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329378 2560 generic.go:356] "Generic (PLEG): container finished" podID="38976ef2-7c34-4df8-9252-337e2e01b774" containerID="33dcbc9e5a0b231162ee018775f572c370610dbbc11dfa4994f325db6d09dc06" exitCode=1 Jul 10 02:23:15.329472 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329448 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"aefd94c89ed2995e85e12148a3658726a9d90166166ab72295834b5cbe1abbc0"} Jul 10 02:23:15.329512 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"9bb54e989e79d537a1fb1324f8f589af88f98583252ed49e569cfa46b1295006"} Jul 10 02:23:15.329596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329548 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"ca66c53290737b8e7bce95850bea0a279ee3038a1d328bceb347d2f885f1fff4"} Jul 10 02:23:15.329596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"4d617d3966f26d97376cbd072c24a4bf97d176d206bc01a3872e72a225114b88"} Jul 10 02:23:15.329674 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.329624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerDied","Data":"33dcbc9e5a0b231162ee018775f572c370610dbbc11dfa4994f325db6d09dc06"} Jul 10 02:23:15.330732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.330707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7lpds" event={"ID":"ab9baa9e-314f-4d93-aba3-0bc72da716c5","Type":"ContainerStarted","Data":"4a48ae1af4b9bded75ae926f2d2569df3dabfcf922ecaca0b8bc3d5b8045e9d7"} Jul 10 02:23:15.331958 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.331934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" event={"ID":"a04013e6-a20f-4770-b890-1c3e32dbe1ad","Type":"ContainerStarted","Data":"9ace52b37c853c5048a788c65bcc079b3be0929ac962fc7783d258cab1385443"} Jul 10 02:23:15.356628 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.356583 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7lpds" podStartSLOduration=3.667888703 podStartE2EDuration="22.356566907s" totalImagesPullingTime="18.688678204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:15.355708669 +0000 UTC m=+22.632498114" watchObservedRunningTime="2026-07-10 02:23:15.356566907 +0000 UTC m=+22.633356356" Jul 10 02:23:15.356713 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.356679 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mszkn" podStartSLOduration=3.6718601079999997 podStartE2EDuration="22.356670316s" totalImagesPullingTime="18.684810208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:15.340097164 +0000 UTC m=+22.616886634" watchObservedRunningTime="2026-07-10 02:23:15.356670316 +0000 UTC m=+22.633459764" Jul 10 02:23:15.409831 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.409780 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:23:15.410322 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.410308 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d7787" Jul 10 02:23:15.687999 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:15.687976 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 02:23:16.132006 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.131665 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:23:15.687995939Z","UUID":"48d29aa1-b41f-4625-b4ff-9413b285a27b","Handler":null,"Name":"","Endpoint":""} Jul 10 02:23:16.133314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.133284 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 02:23:16.133496 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.133344 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 02:23:16.337354 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.337323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" event={"ID":"a04013e6-a20f-4770-b890-1c3e32dbe1ad","Type":"ContainerStarted","Data":"db8b5433b3a8ec85b4da5d1eb489db5aab3743646f9116cec866d2c0126c626f"} Jul 10 02:23:16.339980 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.339897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c4x5k" event={"ID":"3dd8b7cb-f267-40e5-8b83-cba4df3e00b4","Type":"ContainerStarted","Data":"3daea356962bf5aeb36f281eb3a30e48e4840679a2069705e966f70d1d7ead52"} Jul 10 02:23:16.357005 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:16.356954 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c4x5k" podStartSLOduration=4.674356047 podStartE2EDuration="23.356937129s" totalImagesPullingTime="18.682581082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:16.356512205 +0000 UTC m=+23.633301675" watchObservedRunningTime="2026-07-10 02:23:16.356937129 +0000 UTC m=+23.633726576" Jul 10 02:23:17.218793 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.218547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:17.218938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.218591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:17.218938 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:17.218850 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:17.218938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.218591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:17.218938 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:17.218915 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:17.219150 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:17.219018 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:17.344909 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.344881 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:23:17.345338 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.345310 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"b191c6fcf6083b208d7cee09f55923cd6766846545f00a56d60b687ad65294ad"} Jul 10 02:23:17.347495 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.347467 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" event={"ID":"a04013e6-a20f-4770-b890-1c3e32dbe1ad","Type":"ContainerStarted","Data":"27bbfccf496c53583c8231b03b53886f3af2a72efdb282c3021af7e13fd408e2"} Jul 10 02:23:17.367614 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:17.367573 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsztl" podStartSLOduration=3.618765254 podStartE2EDuration="24.367560793s" totalImagesPullingTime="20.748795539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:17.366934668 +0000 UTC m=+24.643724114" watchObservedRunningTime="2026-07-10 02:23:17.367560793 +0000 UTC m=+24.644350238" Jul 10 02:23:19.218538 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.218513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:19.219069 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:19.218642 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:19.219069 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.218972 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:19.219069 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:19.219041 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:19.219194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.219150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:19.219241 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:19.219228 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:19.355680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.355441 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:23:19.356145 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.355809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"04408294d154935e03139ce9fda969379e3f6e4e5426dd14996a2e32476fe295"} Jul 10 02:23:19.356450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.356194 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:19.356450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.356261 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:19.356450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.356363 2560 scope.go:117] "RemoveContainer" containerID="33dcbc9e5a0b231162ee018775f572c370610dbbc11dfa4994f325db6d09dc06" Jul 10 02:23:19.379089 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:19.378996 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:20.359527 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.359284 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="6a5384ab83ae09ff9d883baa0babdca9f1b5f43d73bbe645cac1c438c8e05176" exitCode=0 Jul 10 02:23:20.360045 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.359359 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"6a5384ab83ae09ff9d883baa0babdca9f1b5f43d73bbe645cac1c438c8e05176"} Jul 10 02:23:20.363208 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.363191 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:23:20.363521 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.363498 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" event={"ID":"38976ef2-7c34-4df8-9252-337e2e01b774","Type":"ContainerStarted","Data":"6afc1830babd6105e6ed451894cf1f6890b0607d5611313474351fc6cff84275"} Jul 10 02:23:20.363835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.363817 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:20.378206 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.378187 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:20.412483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:20.412441 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" podStartSLOduration=8.631155233 podStartE2EDuration="27.412428897s" totalImagesPullingTime="18.781273664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:20.412109866 +0000 UTC m=+27.688899331" watchObservedRunningTime="2026-07-10 02:23:20.412428897 +0000 UTC m=+27.689218343" Jul 10 02:23:21.219199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.219012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:21.219333 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.219012 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:21.219333 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.219020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:21.219333 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.219227 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:21.219445 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.219368 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:21.219489 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.219461 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:21.340214 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.340186 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6wg4"] Jul 10 02:23:21.343067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.343033 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gqsgv"] Jul 10 02:23:21.343861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.343833 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4b7g"] Jul 10 02:23:21.368325 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.368290 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="cd390158775d5ce1ba2d9e554b289b3a61179c513e9f669b8a649baad0a9f979" exitCode=0 Jul 10 02:23:21.368687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.368360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"cd390158775d5ce1ba2d9e554b289b3a61179c513e9f669b8a649baad0a9f979"} Jul 10 02:23:21.368687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.368401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:21.368687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.368515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:21.368687 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.368537 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:21.368687 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.368654 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:21.368908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:21.368706 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:21.368908 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:21.368795 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:22.373257 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:22.373175 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="22eb13b3b03c0f8326c4b4c14eea47d7507d2b40fcb7e7ad0df6edd9b4394f5c" exitCode=0 Jul 10 02:23:22.373632 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:22.373250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"22eb13b3b03c0f8326c4b4c14eea47d7507d2b40fcb7e7ad0df6edd9b4394f5c"} Jul 10 02:23:23.219915 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:23.219872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:23.220386 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:23.219979 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:23.220386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:23.220088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:23.220386 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:23.220185 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:23.220386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:23.220232 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:23.220386 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:23.220321 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:25.218822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:25.218784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:25.218822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:25.218805 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:25.219643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:25.218794 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:25.219643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:25.218918 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gqsgv" podUID="79edebb3-eeb7-4fbe-b368-99e83af6b4d1" Jul 10 02:23:25.219643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:25.219012 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6wg4" podUID="179493af-371a-414e-9602-5de5cae4d7a8" Jul 10 02:23:25.219643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:25.219088 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4b7g" podUID="7938b29a-10b1-4d3c-ae49-ba74c70523ac" Jul 10 02:23:26.844509 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:26.844444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:26.844887 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:26.844591 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:26.844887 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:26.844647 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs podName:179493af-371a-414e-9602-5de5cae4d7a8 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:58.844632961 +0000 UTC m=+66.121422390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs") pod "network-metrics-daemon-j6wg4" (UID: "179493af-371a-414e-9602-5de5cae4d7a8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:23:26.945649 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:26.945625 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:26.945772 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:26.945725 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:26.945816 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:26.945775 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret podName:79edebb3-eeb7-4fbe-b368-99e83af6b4d1 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:58.94576209 +0000 UTC m=+66.222551518 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret") pod "global-pull-secret-syncer-gqsgv" (UID: "79edebb3-eeb7-4fbe-b368-99e83af6b4d1") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:23:27.046662 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.046633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:27.046792 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.046774 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:23:27.046837 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.046796 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:23:27.046837 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.046806 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9fkjw for pod openshift-network-diagnostics/network-check-target-d4b7g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:27.046908 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.046853 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw podName:7938b29a-10b1-4d3c-ae49-ba74c70523ac nodeName:}" failed. No retries permitted until 2026-07-10 02:23:59.046840544 +0000 UTC m=+66.323629968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9fkjw" (UniqueName: "kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw") pod "network-check-target-d4b7g" (UID: "7938b29a-10b1-4d3c-ae49-ba74c70523ac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:23:27.062939 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.062919 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-192.ec2.internal" event="NodeReady" Jul 10 02:23:27.063048 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.063014 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 02:23:27.109143 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.109085 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x8hz8"] Jul 10 02:23:27.120401 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.120383 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tbkfz"] Jul 10 02:23:27.120502 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.120458 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.122874 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.122855 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 02:23:27.122874 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.122867 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 02:23:27.123004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.122913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qdhtc\"" Jul 10 02:23:27.132575 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.132556 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8hz8"] Jul 10 02:23:27.132678 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.132593 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tbkfz"] Jul 10 02:23:27.132737 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.132677 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.135079 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.135058 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 02:23:27.135299 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.135283 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 02:23:27.135376 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.135335 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c8vpj\"" Jul 10 02:23:27.135523 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.135511 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 02:23:27.147567 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147546 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375993c8-bcec-4457-b129-456c3fe3b9c2-cert\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.147645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3b7624f-c3b1-4041-b6a1-0202d4772de0-config-volume\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.147645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147599 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3b7624f-c3b1-4041-b6a1-0202d4772de0-tmp-dir\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.147715 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvxmf\" (UniqueName: \"kubernetes.io/projected/375993c8-bcec-4457-b129-456c3fe3b9c2-kube-api-access-fvxmf\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.147715 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147678 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.147776 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.147746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c75k\" (UniqueName: \"kubernetes.io/projected/e3b7624f-c3b1-4041-b6a1-0202d4772de0-kube-api-access-4c75k\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.218845 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.218825 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:27.218976 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.218825 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:27.219020 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.218825 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:27.221727 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.221696 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 02:23:27.221727 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.221705 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r7lb\"" Jul 10 02:23:27.221888 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.221735 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 02:23:27.221888 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.221759 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 02:23:27.221888 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.221825 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 02:23:27.222143 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.222124 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gcczx\"" Jul 10 02:23:27.248416 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375993c8-bcec-4457-b129-456c3fe3b9c2-cert\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.248501 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3b7624f-c3b1-4041-b6a1-0202d4772de0-config-volume\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.248501 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3b7624f-c3b1-4041-b6a1-0202d4772de0-tmp-dir\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.248501 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvxmf\" (UniqueName: \"kubernetes.io/projected/375993c8-bcec-4457-b129-456c3fe3b9c2-kube-api-access-fvxmf\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.248617 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.248617 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c75k\" (UniqueName: \"kubernetes.io/projected/e3b7624f-c3b1-4041-b6a1-0202d4772de0-kube-api-access-4c75k\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.248729 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.248656 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:27.248729 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.248723 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:27.748702363 +0000 UTC m=+35.025491799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:27.249014 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3b7624f-c3b1-4041-b6a1-0202d4772de0-tmp-dir\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.249060 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.248974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3b7624f-c3b1-4041-b6a1-0202d4772de0-config-volume\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.252584 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.252562 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/375993c8-bcec-4457-b129-456c3fe3b9c2-cert\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.262842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.262823 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c75k\" (UniqueName: \"kubernetes.io/projected/e3b7624f-c3b1-4041-b6a1-0202d4772de0-kube-api-access-4c75k\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.262928 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.262862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvxmf\" (UniqueName: \"kubernetes.io/projected/375993c8-bcec-4457-b129-456c3fe3b9c2-kube-api-access-fvxmf\") pod \"ingress-canary-tbkfz\" (UID: \"375993c8-bcec-4457-b129-456c3fe3b9c2\") " pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.388031 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.387960 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="67bbe82c45aa75769f7a01e074c93fb6c98ae6adf5113352a9da7896a3d62fac" exitCode=0 Jul 10 02:23:27.388031 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.388011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"67bbe82c45aa75769f7a01e074c93fb6c98ae6adf5113352a9da7896a3d62fac"} Jul 10 02:23:27.440893 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.440872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tbkfz" Jul 10 02:23:27.600050 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.600021 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tbkfz"] Jul 10 02:23:27.605858 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:27.605824 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375993c8_bcec_4457_b129_456c3fe3b9c2.slice/crio-8a7994ca61bc46ee434b78191115e3267cffe01caf84b6c44afcbd312d2feced WatchSource:0}: Error finding container 8a7994ca61bc46ee434b78191115e3267cffe01caf84b6c44afcbd312d2feced: Status 404 returned error can't find the container with id 8a7994ca61bc46ee434b78191115e3267cffe01caf84b6c44afcbd312d2feced Jul 10 02:23:27.751967 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:27.751917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:27.752106 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.752030 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:27.752106 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:27.752074 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:28.752061464 +0000 UTC m=+36.028850888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:28.289779 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.289738 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g"] Jul 10 02:23:28.307655 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.307631 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g"] Jul 10 02:23:28.307801 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.307737 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.310150 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.310120 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 02:23:28.310150 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.310147 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 02:23:28.311013 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.310994 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 02:23:28.311137 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.311038 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dl77q\"" Jul 10 02:23:28.311137 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.311043 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:23:28.356889 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.356863 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn4mp\" (UniqueName: \"kubernetes.io/projected/16df9165-f721-462a-b576-e6c4f2bf5143-kube-api-access-bn4mp\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.357012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.356984 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16df9165-f721-462a-b576-e6c4f2bf5143-config\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.357069 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.357018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16df9165-f721-462a-b576-e6c4f2bf5143-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.392687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.392647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tbkfz" event={"ID":"375993c8-bcec-4457-b129-456c3fe3b9c2","Type":"ContainerStarted","Data":"8a7994ca61bc46ee434b78191115e3267cffe01caf84b6c44afcbd312d2feced"} Jul 10 02:23:28.393684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.393661 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx"] Jul 10 02:23:28.396815 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.396789 2560 generic.go:356] "Generic (PLEG): container finished" podID="34b111d8-1f9a-41a2-baf7-2ad06a1a04a9" containerID="140b06cf83a1f94389425ae95513b934c9905de803cb045f573c1c28e70b64f5" exitCode=0 Jul 10 02:23:28.424451 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.424426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerDied","Data":"140b06cf83a1f94389425ae95513b934c9905de803cb045f573c1c28e70b64f5"} Jul 10 02:23:28.424550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.424469 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t"] Jul 10 02:23:28.424550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.424543 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" Jul 10 02:23:28.428074 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.428054 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-8cfgf\"" Jul 10 02:23:28.440366 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.440338 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:23:28.440481 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.440455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.444929 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.444911 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 02:23:28.445022 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.444959 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 02:23:28.447669 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.447498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 02:23:28.448339 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.448137 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:23:28.450475 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.449573 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qcsz9\"" Jul 10 02:23:28.458337 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16df9165-f721-462a-b576-e6c4f2bf5143-config\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.458450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458373 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16df9165-f721-462a-b576-e6c4f2bf5143-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.458450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q67qd\" (UniqueName: \"kubernetes.io/projected/f4c954d2-f958-4402-8129-3d8d188b5d1e-kube-api-access-q67qd\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.458564 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4c954d2-f958-4402-8129-3d8d188b5d1e-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.458623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn4mp\" (UniqueName: \"kubernetes.io/projected/16df9165-f721-462a-b576-e6c4f2bf5143-kube-api-access-bn4mp\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.458704 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458688 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xct4l\" (UniqueName: \"kubernetes.io/projected/5238fb47-245e-4e25-b779-f2230f006f56-kube-api-access-xct4l\") pod \"network-check-source-869dcfcdcd-ksvnx\" (UID: \"5238fb47-245e-4e25-b779-f2230f006f56\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" Jul 10 02:23:28.458859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.458843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c954d2-f958-4402-8129-3d8d188b5d1e-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.460347 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.459994 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16df9165-f721-462a-b576-e6c4f2bf5143-config\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.464911 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.464725 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16df9165-f721-462a-b576-e6c4f2bf5143-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.469239 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.469213 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn4mp\" (UniqueName: \"kubernetes.io/projected/16df9165-f721-462a-b576-e6c4f2bf5143-kube-api-access-bn4mp\") pod \"service-ca-operator-7695fc6c8c-jnk2g\" (UID: \"16df9165-f721-462a-b576-e6c4f2bf5143\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.472952 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.472781 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx"] Jul 10 02:23:28.472952 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.472818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t"] Jul 10 02:23:28.472952 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.472840 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:23:28.472952 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.472931 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.475809 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.475787 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 02:23:28.476068 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.476050 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zf7mn\"" Jul 10 02:23:28.476247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.476230 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 02:23:28.476943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.476658 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 02:23:28.488645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.488624 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 02:23:28.559352 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559352 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q67qd\" (UniqueName: \"kubernetes.io/projected/f4c954d2-f958-4402-8129-3d8d188b5d1e-kube-api-access-q67qd\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4c954d2-f958-4402-8129-3d8d188b5d1e-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559438 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9lxl\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xct4l\" (UniqueName: \"kubernetes.io/projected/5238fb47-245e-4e25-b779-f2230f006f56-kube-api-access-xct4l\") pod \"network-check-source-869dcfcdcd-ksvnx\" (UID: \"5238fb47-245e-4e25-b779-f2230f006f56\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" Jul 10 02:23:28.559862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c954d2-f958-4402-8129-3d8d188b5d1e-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.559862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.559862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.559683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.560094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.560072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f4c954d2-f958-4402-8129-3d8d188b5d1e-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.562376 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.562352 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f4c954d2-f958-4402-8129-3d8d188b5d1e-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.569415 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.569394 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q67qd\" (UniqueName: \"kubernetes.io/projected/f4c954d2-f958-4402-8129-3d8d188b5d1e-kube-api-access-q67qd\") pod \"kube-storage-version-migrator-operator-799cb7f958-vbp4t\" (UID: \"f4c954d2-f958-4402-8129-3d8d188b5d1e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.569496 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.569472 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xct4l\" (UniqueName: \"kubernetes.io/projected/5238fb47-245e-4e25-b779-f2230f006f56-kube-api-access-xct4l\") pod \"network-check-source-869dcfcdcd-ksvnx\" (UID: \"5238fb47-245e-4e25-b779-f2230f006f56\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" Jul 10 02:23:28.619534 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.619509 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" Jul 10 02:23:28.661007 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.660973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661142 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9lxl\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661202 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661165 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661202 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661322 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661322 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661262 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661322 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661463 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.661329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.661463 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:28.661382 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:28.661463 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:28.661397 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:28.661602 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:28.661466 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:23:29.16143375 +0000 UTC m=+36.438223176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:28.662361 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.662316 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.662738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.662713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.662825 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.662795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.664195 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.664175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.664312 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.664256 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.680983 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.680928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9lxl\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.690170 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.690128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:28.735823 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.735796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" Jul 10 02:23:28.760179 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.760155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" Jul 10 02:23:28.762544 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.762523 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:28.762667 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:28.762653 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:28.762763 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:28.762708 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:30.762694759 +0000 UTC m=+38.039484182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:28.774579 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.774551 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g"] Jul 10 02:23:28.778341 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:28.778317 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16df9165_f721_462a_b576_e6c4f2bf5143.slice/crio-9cce84e63d9ae1ca379164d201d1a1740cfda12332c3746e6dfd62bb13a1147b WatchSource:0}: Error finding container 9cce84e63d9ae1ca379164d201d1a1740cfda12332c3746e6dfd62bb13a1147b: Status 404 returned error can't find the container with id 9cce84e63d9ae1ca379164d201d1a1740cfda12332c3746e6dfd62bb13a1147b Jul 10 02:23:28.922016 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.921988 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx"] Jul 10 02:23:28.924918 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:28.924890 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5238fb47_245e_4e25_b779_f2230f006f56.slice/crio-40b9e94be67ecc2f58a325b00cb86044b8af7e558dc530cf21e48bc2c29d61a6 WatchSource:0}: Error finding container 40b9e94be67ecc2f58a325b00cb86044b8af7e558dc530cf21e48bc2c29d61a6: Status 404 returned error can't find the container with id 40b9e94be67ecc2f58a325b00cb86044b8af7e558dc530cf21e48bc2c29d61a6 Jul 10 02:23:28.939498 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:28.939478 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t"] Jul 10 02:23:28.943736 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:28.943708 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c954d2_f958_4402_8129_3d8d188b5d1e.slice/crio-60e9a5786942f91e973450c01f044eb7fcd3ddbc965b984f1e40f57e832ffe63 WatchSource:0}: Error finding container 60e9a5786942f91e973450c01f044eb7fcd3ddbc965b984f1e40f57e832ffe63: Status 404 returned error can't find the container with id 60e9a5786942f91e973450c01f044eb7fcd3ddbc965b984f1e40f57e832ffe63 Jul 10 02:23:29.165842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.165771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:29.165979 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:29.165926 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:29.165979 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:29.165944 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:29.166070 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:29.166004 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:23:30.165984209 +0000 UTC m=+37.442773652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:29.403621 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.403550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" event={"ID":"34b111d8-1f9a-41a2-baf7-2ad06a1a04a9","Type":"ContainerStarted","Data":"62181ac7893dda3c9ae8ce3c5c0f84145e426a08891f5bc763f495c3ac402e66"} Jul 10 02:23:29.404922 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.404891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" event={"ID":"5238fb47-245e-4e25-b779-f2230f006f56","Type":"ContainerStarted","Data":"40b9e94be67ecc2f58a325b00cb86044b8af7e558dc530cf21e48bc2c29d61a6"} Jul 10 02:23:29.406118 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.406091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" event={"ID":"16df9165-f721-462a-b576-e6c4f2bf5143","Type":"ContainerStarted","Data":"9cce84e63d9ae1ca379164d201d1a1740cfda12332c3746e6dfd62bb13a1147b"} Jul 10 02:23:29.407202 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.407179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" event={"ID":"f4c954d2-f958-4402-8129-3d8d188b5d1e","Type":"ContainerStarted","Data":"60e9a5786942f91e973450c01f044eb7fcd3ddbc965b984f1e40f57e832ffe63"} Jul 10 02:23:29.429748 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:29.429658 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v8sj7" podStartSLOduration=8.309029919 podStartE2EDuration="36.429639533s" totalImagesPullingTime="28.120609614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:29.42946874 +0000 UTC m=+36.706258211" watchObservedRunningTime="2026-07-10 02:23:29.429639533 +0000 UTC m=+36.706428981" Jul 10 02:23:30.174577 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:30.174550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:30.174710 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:30.174695 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:30.174757 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:30.174714 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:30.174804 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:30.174772 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:23:32.174754813 +0000 UTC m=+39.451544242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:30.413483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:30.413160 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tbkfz" event={"ID":"375993c8-bcec-4457-b129-456c3fe3b9c2","Type":"ContainerStarted","Data":"d4a875d276ba608e11e46c536f1519c5ff64e6966656c0c808c8fd28259a60c9"} Jul 10 02:23:30.437206 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:30.436150 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tbkfz" podStartSLOduration=0.948816214 podStartE2EDuration="3.436130014s" totalImagesPullingTime="2.4873138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:30.432129308 +0000 UTC m=+37.708918780" watchObservedRunningTime="2026-07-10 02:23:30.436130014 +0000 UTC m=+37.712919462" Jul 10 02:23:30.781372 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:30.780739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:30.781372 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:30.780943 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:30.781372 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:30.781003 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:34.780984803 +0000 UTC m=+42.057774232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:32.191341 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:32.191306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:32.191726 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:32.191449 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:32.191726 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:32.191461 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:32.191726 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:32.191515 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:23:36.191499329 +0000 UTC m=+43.468288754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:34.355468 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.355432 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hlp29"] Jul 10 02:23:34.380732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.380708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.384926 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.384902 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 02:23:34.385559 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.385519 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 02:23:34.387640 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.387619 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pfnlm\"" Jul 10 02:23:34.388333 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.388104 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 02:23:34.389498 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.389479 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 02:23:34.395287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.395254 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hlp29"] Jul 10 02:23:34.408832 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.408808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.408948 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.408845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlnkb\" (UniqueName: \"kubernetes.io/projected/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-api-access-wlnkb\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.409037 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.409015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-data-volume\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.409091 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.409051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.409142 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.409086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-crio-socket\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.424639 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.424614 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" event={"ID":"16df9165-f721-462a-b576-e6c4f2bf5143","Type":"ContainerStarted","Data":"e7406304e2f976223de407c57150d4cafdaba267b4cd297a1f58efc603887f81"} Jul 10 02:23:34.426435 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.426396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" event={"ID":"f4c954d2-f958-4402-8129-3d8d188b5d1e","Type":"ContainerStarted","Data":"c81be6fcf8260adebb1c6d69c98d4b53c1d3acc9272f6cab6493b804424ce5d1"} Jul 10 02:23:34.428237 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.428208 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" event={"ID":"5238fb47-245e-4e25-b779-f2230f006f56","Type":"ContainerStarted","Data":"c3e9329e85f20a10965c60a3fac6919f3a8c2c1c4fd6fc3b037fd9ccdad3a4b3"} Jul 10 02:23:34.451300 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.451210 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" podStartSLOduration=1.084916341 podStartE2EDuration="6.451192702s" totalImagesPullingTime="5.366276361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:34.450592935 +0000 UTC m=+41.727382405" watchObservedRunningTime="2026-07-10 02:23:34.451192702 +0000 UTC m=+41.727982150" Jul 10 02:23:34.471152 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.471115 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" podStartSLOduration=1.270044616 podStartE2EDuration="6.471101965s" totalImagesPullingTime="5.201057349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:34.470223769 +0000 UTC m=+41.747013215" watchObservedRunningTime="2026-07-10 02:23:34.471101965 +0000 UTC m=+41.747891420" Jul 10 02:23:34.487420 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.487386 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksvnx" podStartSLOduration=1.259440416 podStartE2EDuration="6.487379409s" totalImagesPullingTime="5.227938993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:34.486286454 +0000 UTC m=+41.763075916" watchObservedRunningTime="2026-07-10 02:23:34.487379409 +0000 UTC m=+41.764168854" Jul 10 02:23:34.509930 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.509906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-data-volume\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510040 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.509942 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510040 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.509974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-crio-socket\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510040 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.510025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510204 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.510049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlnkb\" (UniqueName: \"kubernetes.io/projected/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-api-access-wlnkb\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510460 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.510439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-crio-socket\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510829 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.510814 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:23:34.510896 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.510851 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-data-volume\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.510896 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.510874 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:35.010857064 +0000 UTC m=+42.287646494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : configmap "kube-rbac-proxy" not found Jul 10 02:23:34.511036 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.510915 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:34.511036 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.510960 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:35.010945397 +0000 UTC m=+42.287734828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:34.520356 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.520329 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlnkb\" (UniqueName: \"kubernetes.io/projected/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-api-access-wlnkb\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:34.812667 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:34.812633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:34.812815 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.812788 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:34.812879 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:34.812862 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:42.812841045 +0000 UTC m=+50.089630476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:35.014540 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:35.014514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:35.014698 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:35.014554 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:35.014698 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:35.014652 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:23:35.014698 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:35.014663 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:35.014842 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:35.014708 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:36.014691119 +0000 UTC m=+43.291480542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:35.014842 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:35.014739 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:36.014717268 +0000 UTC m=+43.291506693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : configmap "kube-rbac-proxy" not found Jul 10 02:23:36.023230 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.023196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:36.023686 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.023349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:36.023686 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.023378 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:36.023686 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.023439 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:23:36.023686 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.023443 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:38.023426459 +0000 UTC m=+45.300215889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:36.023686 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.023498 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:38.023481547 +0000 UTC m=+45.300270971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : configmap "kube-rbac-proxy" not found Jul 10 02:23:36.166870 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.166842 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lpds_ab9baa9e-314f-4d93-aba3-0bc72da716c5/dns-node-resolver/0.log" Jul 10 02:23:36.224551 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.224522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:36.224677 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.224666 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:36.224719 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.224681 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:36.224755 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:36.224731 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:23:44.2247151 +0000 UTC m=+51.501504529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:36.655638 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.655608 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz"] Jul 10 02:23:36.687043 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.687011 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz"] Jul 10 02:23:36.687196 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.687153 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" Jul 10 02:23:36.689748 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.689708 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 02:23:36.689882 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.689798 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 02:23:36.689882 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.689824 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ltqhj\"" Jul 10 02:23:36.729026 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.729001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz9gx\" (UniqueName: \"kubernetes.io/projected/73d6d0fb-a0c8-4f23-bab3-806aca111d94-kube-api-access-qz9gx\") pod \"migrator-7f59b54f6c-pqhbz\" (UID: \"73d6d0fb-a0c8-4f23-bab3-806aca111d94\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" Jul 10 02:23:36.829463 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.829429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz9gx\" (UniqueName: \"kubernetes.io/projected/73d6d0fb-a0c8-4f23-bab3-806aca111d94-kube-api-access-qz9gx\") pod \"migrator-7f59b54f6c-pqhbz\" (UID: \"73d6d0fb-a0c8-4f23-bab3-806aca111d94\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" Jul 10 02:23:36.837367 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.837344 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz9gx\" (UniqueName: \"kubernetes.io/projected/73d6d0fb-a0c8-4f23-bab3-806aca111d94-kube-api-access-qz9gx\") pod \"migrator-7f59b54f6c-pqhbz\" (UID: \"73d6d0fb-a0c8-4f23-bab3-806aca111d94\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" Jul 10 02:23:36.996399 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:36.996349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" Jul 10 02:23:37.146372 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:37.146343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz"] Jul 10 02:23:37.152201 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:37.152165 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d6d0fb_a0c8_4f23_bab3_806aca111d94.slice/crio-15533e37e0250a68eed2c5f878318478beb447cabfc1d378c81cf4e556a2b946 WatchSource:0}: Error finding container 15533e37e0250a68eed2c5f878318478beb447cabfc1d378c81cf4e556a2b946: Status 404 returned error can't find the container with id 15533e37e0250a68eed2c5f878318478beb447cabfc1d378c81cf4e556a2b946 Jul 10 02:23:37.436683 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:37.436650 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" event={"ID":"73d6d0fb-a0c8-4f23-bab3-806aca111d94","Type":"ContainerStarted","Data":"15533e37e0250a68eed2c5f878318478beb447cabfc1d378c81cf4e556a2b946"} Jul 10 02:23:37.567596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:37.567572 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mszkn_6d4ef782-8911-480a-90dd-63d7de6699f3/node-ca/0.log" Jul 10 02:23:38.037937 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:38.037904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:38.038103 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:38.037958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:38.038103 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:38.038025 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:23:38.038103 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:38.038055 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:38.038103 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:38.038088 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:42.038071974 +0000 UTC m=+49.314861409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : configmap "kube-rbac-proxy" not found Jul 10 02:23:38.038103 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:38.038102 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:42.038096006 +0000 UTC m=+49.314885430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:38.368015 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:38.367898 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tbkfz_375993c8-bcec-4457-b129-456c3fe3b9c2/serve-healthcheck-canary/0.log" Jul 10 02:23:39.444447 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:39.444420 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" event={"ID":"73d6d0fb-a0c8-4f23-bab3-806aca111d94","Type":"ContainerStarted","Data":"9681bcc246c2ab77a39fd61eea582a13d475cec0722069238897083b6eb6eaed"} Jul 10 02:23:39.444825 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:39.444468 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" event={"ID":"73d6d0fb-a0c8-4f23-bab3-806aca111d94","Type":"ContainerStarted","Data":"211cbba9a31933b4e1e6c28705a0184859d0f935ccc0b556f14baa7225c5ae77"} Jul 10 02:23:39.464151 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:39.464111 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-pqhbz" podStartSLOduration=1.689026175 podStartE2EDuration="3.464096245s" totalImagesPullingTime="1.77507007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:39.462395957 +0000 UTC m=+46.739185402" watchObservedRunningTime="2026-07-10 02:23:39.464096245 +0000 UTC m=+46.740885690" Jul 10 02:23:42.065925 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:42.065888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:42.066351 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.066028 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:42.066351 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:42.066036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:42.066351 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.066082 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:50.066069324 +0000 UTC m=+57.342858752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:42.066351 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.066096 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:23:42.066351 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.066139 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:50.066127978 +0000 UTC m=+57.342917402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : configmap "kube-rbac-proxy" not found Jul 10 02:23:42.872287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:42.872241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:42.872446 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.872379 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:23:42.872446 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:42.872437 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls podName:e3b7624f-c3b1-4041-b6a1-0202d4772de0 nodeName:}" failed. No retries permitted until 2026-07-10 02:23:58.872422561 +0000 UTC m=+66.149211989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls") pod "dns-default-x8hz8" (UID: "e3b7624f-c3b1-4041-b6a1-0202d4772de0") : secret "dns-default-metrics-tls" not found Jul 10 02:23:44.281946 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:44.281915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:23:44.282389 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:44.282030 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:23:44.282389 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:44.282041 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54667b856c-j2gv6: secret "image-registry-tls" not found Jul 10 02:23:44.282389 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:44.282090 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls podName:79581e9e-3894-4314-b45b-df65e181d71e nodeName:}" failed. No retries permitted until 2026-07-10 02:24:00.282074689 +0000 UTC m=+67.558864114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls") pod "image-registry-54667b856c-j2gv6" (UID: "79581e9e-3894-4314-b45b-df65e181d71e") : secret "image-registry-tls" not found Jul 10 02:23:50.125167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:50.125130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:50.125627 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:50.125218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:50.125627 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:50.125283 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:23:50.125627 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:23:50.125349 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls podName:0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0 nodeName:}" failed. No retries permitted until 2026-07-10 02:24:06.125335168 +0000 UTC m=+73.402124592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlp29" (UID: "0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0") : secret "insights-runtime-extractor-tls" not found Jul 10 02:23:50.125734 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:50.125683 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:23:52.383683 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:52.383656 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htnk4" Jul 10 02:23:58.885665 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.885626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:58.886049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.885725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:58.887886 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.887855 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3b7624f-c3b1-4041-b6a1-0202d4772de0-metrics-tls\") pod \"dns-default-x8hz8\" (UID: \"e3b7624f-c3b1-4041-b6a1-0202d4772de0\") " pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:58.888559 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.888544 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 02:23:58.898491 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.898470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/179493af-371a-414e-9602-5de5cae4d7a8-metrics-certs\") pod \"network-metrics-daemon-j6wg4\" (UID: \"179493af-371a-414e-9602-5de5cae4d7a8\") " pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:58.932101 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.932080 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qdhtc\"" Jul 10 02:23:58.940427 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.940413 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8hz8" Jul 10 02:23:58.986662 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.986621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:58.989792 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:58.989767 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 02:23:59.002016 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.001971 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79edebb3-eeb7-4fbe-b368-99e83af6b4d1-original-pull-secret\") pod \"global-pull-secret-syncer-gqsgv\" (UID: \"79edebb3-eeb7-4fbe-b368-99e83af6b4d1\") " pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:59.028450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.028260 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gqsgv" Jul 10 02:23:59.036256 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.036234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gcczx\"" Jul 10 02:23:59.043673 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.043648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6wg4" Jul 10 02:23:59.063792 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.063759 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8hz8"] Jul 10 02:23:59.067910 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:59.067879 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b7624f_c3b1_4041_b6a1_0202d4772de0.slice/crio-1ad2f432dd3c547ebee74f348c70227c5c3468733cafa2874728dd53df1b8839 WatchSource:0}: Error finding container 1ad2f432dd3c547ebee74f348c70227c5c3468733cafa2874728dd53df1b8839: Status 404 returned error can't find the container with id 1ad2f432dd3c547ebee74f348c70227c5c3468733cafa2874728dd53df1b8839 Jul 10 02:23:59.087751 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.087253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:59.090201 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.090179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkjw\" (UniqueName: \"kubernetes.io/projected/7938b29a-10b1-4d3c-ae49-ba74c70523ac-kube-api-access-9fkjw\") pod \"network-check-target-d4b7g\" (UID: \"7938b29a-10b1-4d3c-ae49-ba74c70523ac\") " pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:59.156327 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.156216 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gqsgv"] Jul 10 02:23:59.158817 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:59.158792 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79edebb3_eeb7_4fbe_b368_99e83af6b4d1.slice/crio-5752d8a3d7f1e252aafc0534ff3e35677f59be878a5981914061f777db9ab64a WatchSource:0}: Error finding container 5752d8a3d7f1e252aafc0534ff3e35677f59be878a5981914061f777db9ab64a: Status 404 returned error can't find the container with id 5752d8a3d7f1e252aafc0534ff3e35677f59be878a5981914061f777db9ab64a Jul 10 02:23:59.172852 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.172825 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6wg4"] Jul 10 02:23:59.176662 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:59.176642 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179493af_371a_414e_9602_5de5cae4d7a8.slice/crio-9fbedeae1caa96aa274d16254a7ad5b957840eb6cddb9a72f6158be0943ca9d2 WatchSource:0}: Error finding container 9fbedeae1caa96aa274d16254a7ad5b957840eb6cddb9a72f6158be0943ca9d2: Status 404 returned error can't find the container with id 9fbedeae1caa96aa274d16254a7ad5b957840eb6cddb9a72f6158be0943ca9d2 Jul 10 02:23:59.340361 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.340335 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r7lb\"" Jul 10 02:23:59.348523 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.348505 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:23:59.479854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.479826 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4b7g"] Jul 10 02:23:59.483369 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:23:59.483341 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7938b29a_10b1_4d3c_ae49_ba74c70523ac.slice/crio-615628e7e23d1f153649cb66eba65b89dacdd028e3da3119731b41d24f6de829 WatchSource:0}: Error finding container 615628e7e23d1f153649cb66eba65b89dacdd028e3da3119731b41d24f6de829: Status 404 returned error can't find the container with id 615628e7e23d1f153649cb66eba65b89dacdd028e3da3119731b41d24f6de829 Jul 10 02:23:59.505607 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.505554 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4b7g" event={"ID":"7938b29a-10b1-4d3c-ae49-ba74c70523ac","Type":"ContainerStarted","Data":"615628e7e23d1f153649cb66eba65b89dacdd028e3da3119731b41d24f6de829"} Jul 10 02:23:59.506836 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.506806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6wg4" event={"ID":"179493af-371a-414e-9602-5de5cae4d7a8","Type":"ContainerStarted","Data":"9fbedeae1caa96aa274d16254a7ad5b957840eb6cddb9a72f6158be0943ca9d2"} Jul 10 02:23:59.508282 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.508244 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gqsgv" event={"ID":"79edebb3-eeb7-4fbe-b368-99e83af6b4d1","Type":"ContainerStarted","Data":"5752d8a3d7f1e252aafc0534ff3e35677f59be878a5981914061f777db9ab64a"} Jul 10 02:23:59.509548 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:23:59.509496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8hz8" event={"ID":"e3b7624f-c3b1-4041-b6a1-0202d4772de0","Type":"ContainerStarted","Data":"1ad2f432dd3c547ebee74f348c70227c5c3468733cafa2874728dd53df1b8839"} Jul 10 02:24:00.297082 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.297050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:00.300058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.300030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"image-registry-54667b856c-j2gv6\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:00.516018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.515970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4b7g" event={"ID":"7938b29a-10b1-4d3c-ae49-ba74c70523ac","Type":"ContainerStarted","Data":"d2dfa88f44a2f92bd0210a6899229280397bd09d26d72fc08455835bf2b1fe08"} Jul 10 02:24:00.516188 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.516043 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:24:00.532746 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.532693 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d4b7g" podStartSLOduration=67.532675549 podStartE2EDuration="1m7.532675549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:00.531892573 +0000 UTC m=+67.808682022" watchObservedRunningTime="2026-07-10 02:24:00.532675549 +0000 UTC m=+67.809464996" Jul 10 02:24:00.593359 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.593284 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zf7mn\"" Jul 10 02:24:00.600909 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.600524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:00.872403 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:00.872256 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:24:00.877433 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:00.877399 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79581e9e_3894_4314_b45b_df65e181d71e.slice/crio-3db66b714608cd3a9d7913118b9c9c2abfeffd3b4d3093b698e2e962ecc904f5 WatchSource:0}: Error finding container 3db66b714608cd3a9d7913118b9c9c2abfeffd3b4d3093b698e2e962ecc904f5: Status 404 returned error can't find the container with id 3db66b714608cd3a9d7913118b9c9c2abfeffd3b4d3093b698e2e962ecc904f5 Jul 10 02:24:01.522512 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.522472 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6wg4" event={"ID":"179493af-371a-414e-9602-5de5cae4d7a8","Type":"ContainerStarted","Data":"c1fef5d6ea457dfb8fefb351a983d91eb4235ace6032256867228a2e37397e2c"} Jul 10 02:24:01.523012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.522539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6wg4" event={"ID":"179493af-371a-414e-9602-5de5cae4d7a8","Type":"ContainerStarted","Data":"7e9a1d521141365a0a6935fd0385e06ec4b413e15283b3b275d6b43107fcb26d"} Jul 10 02:24:01.524468 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.524441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8hz8" event={"ID":"e3b7624f-c3b1-4041-b6a1-0202d4772de0","Type":"ContainerStarted","Data":"ddc6e2133f2039e38e3b42b5d2d11d05e8c48f5570b35330cd74ffe46815b869"} Jul 10 02:24:01.524596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.524492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8hz8" event={"ID":"e3b7624f-c3b1-4041-b6a1-0202d4772de0","Type":"ContainerStarted","Data":"6efc22f37b71c3ffa44cc675928c00d0865ebf07aae00fe458cf677d60b14b20"} Jul 10 02:24:01.524596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.524518 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x8hz8" Jul 10 02:24:01.526132 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.526068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" event={"ID":"79581e9e-3894-4314-b45b-df65e181d71e","Type":"ContainerStarted","Data":"2c45bdf7739a2209ca21e8d020ffdaf847167e0a4f0e09388c12846fd4abcd76"} Jul 10 02:24:01.526132 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.526118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" event={"ID":"79581e9e-3894-4314-b45b-df65e181d71e","Type":"ContainerStarted","Data":"3db66b714608cd3a9d7913118b9c9c2abfeffd3b4d3093b698e2e962ecc904f5"} Jul 10 02:24:01.540718 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.540669 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j6wg4" podStartSLOduration=66.995564846 podStartE2EDuration="1m8.54065113s" totalImagesPullingTime="1.545086284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:01.538966352 +0000 UTC m=+68.815755824" watchObservedRunningTime="2026-07-10 02:24:01.54065113 +0000 UTC m=+68.817440577" Jul 10 02:24:01.558091 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.557968 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x8hz8" podStartSLOduration=32.854404427 podStartE2EDuration="34.55795056s" totalImagesPullingTime="1.703546133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:01.55714224 +0000 UTC m=+68.833931687" watchObservedRunningTime="2026-07-10 02:24:01.55795056 +0000 UTC m=+68.834740018" Jul 10 02:24:01.578106 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.578069 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" podStartSLOduration=33.578056083999996 podStartE2EDuration="33.578056084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:01.576972689 +0000 UTC m=+68.853762125" watchObservedRunningTime="2026-07-10 02:24:01.578056084 +0000 UTC m=+68.854845529" Jul 10 02:24:01.877844 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.877744 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt"] Jul 10 02:24:01.881255 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.881225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:01.882508 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.882483 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:24:01.884619 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.884596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 02:24:01.884751 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.884721 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 02:24:01.884816 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.884779 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 02:24:01.884869 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.884832 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 02:24:01.884926 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.884896 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xdfqz\"" Jul 10 02:24:01.894025 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.893993 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt"] Jul 10 02:24:01.982429 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.982398 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-749ffcb46b-cndhj"] Jul 10 02:24:01.985511 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.985493 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:01.999483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:01.999464 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-749ffcb46b-cndhj"] Jul 10 02:24:02.013943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.013912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e627d619-379f-4409-9282-fdcc9d4ec9ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.014100 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.013964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45z7s\" (UniqueName: \"kubernetes.io/projected/e627d619-379f-4409-9282-fdcc9d4ec9ef-kube-api-access-45z7s\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.114748 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114718 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-trusted-ca\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.114943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-registry-tls\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.114943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24qmh\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-kube-api-access-24qmh\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.114943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-bound-sa-token\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.114943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-registry-certificates\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.114943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-installation-pull-secrets\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.115171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.114967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e627d619-379f-4409-9282-fdcc9d4ec9ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.115171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.115035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45z7s\" (UniqueName: \"kubernetes.io/projected/e627d619-379f-4409-9282-fdcc9d4ec9ef-kube-api-access-45z7s\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.115171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.115067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04716cff-18b9-4372-92d0-444f4b236c51-ca-trust-extracted\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.115171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.115104 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-image-registry-private-configuration\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.117738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.117718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e627d619-379f-4409-9282-fdcc9d4ec9ef-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.124711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.124666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45z7s\" (UniqueName: \"kubernetes.io/projected/e627d619-379f-4409-9282-fdcc9d4ec9ef-kube-api-access-45z7s\") pod \"managed-serviceaccount-addon-agent-7fb9468594-vbpdt\" (UID: \"e627d619-379f-4409-9282-fdcc9d4ec9ef\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.201755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.201681 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" Jul 10 02:24:02.216018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.215996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24qmh\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-kube-api-access-24qmh\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216125 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-bound-sa-token\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216125 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216113 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-registry-certificates\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216236 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-installation-pull-secrets\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216236 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04716cff-18b9-4372-92d0-444f4b236c51-ca-trust-extracted\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216360 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-image-registry-private-configuration\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216360 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-trusted-ca\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.216360 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.216351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-registry-tls\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.217129 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.217082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-registry-certificates\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.217525 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.217507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04716cff-18b9-4372-92d0-444f4b236c51-ca-trust-extracted\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.218014 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.217966 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04716cff-18b9-4372-92d0-444f4b236c51-trusted-ca\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.219051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.219015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-installation-pull-secrets\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.219308 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.219254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04716cff-18b9-4372-92d0-444f4b236c51-image-registry-private-configuration\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.219414 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.219358 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-registry-tls\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.226124 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.226076 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-bound-sa-token\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.228041 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.228022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24qmh\" (UniqueName: \"kubernetes.io/projected/04716cff-18b9-4372-92d0-444f4b236c51-kube-api-access-24qmh\") pod \"image-registry-749ffcb46b-cndhj\" (UID: \"04716cff-18b9-4372-92d0-444f4b236c51\") " pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.296719 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.296684 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:02.531251 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:02.530753 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:03.253165 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.253134 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-749ffcb46b-cndhj"] Jul 10 02:24:03.256169 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:03.256144 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04716cff_18b9_4372_92d0_444f4b236c51.slice/crio-ff3fbe9e6fe936096fc6a9c4c94f33410aff9d2b19bfc05eb05ac0c48691fb47 WatchSource:0}: Error finding container ff3fbe9e6fe936096fc6a9c4c94f33410aff9d2b19bfc05eb05ac0c48691fb47: Status 404 returned error can't find the container with id ff3fbe9e6fe936096fc6a9c4c94f33410aff9d2b19bfc05eb05ac0c48691fb47 Jul 10 02:24:03.269238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.269217 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt"] Jul 10 02:24:03.273299 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:03.273252 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode627d619_379f_4409_9282_fdcc9d4ec9ef.slice/crio-1aaadc3f087aa78d64d4232236da55100ecc3254bc08595860e4bd562f007630 WatchSource:0}: Error finding container 1aaadc3f087aa78d64d4232236da55100ecc3254bc08595860e4bd562f007630: Status 404 returned error can't find the container with id 1aaadc3f087aa78d64d4232236da55100ecc3254bc08595860e4bd562f007630 Jul 10 02:24:03.535884 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.535850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gqsgv" event={"ID":"79edebb3-eeb7-4fbe-b368-99e83af6b4d1","Type":"ContainerStarted","Data":"6b748808d4be881d463dcb995ac966bcd3511ffb1addc719b606011d7826b870"} Jul 10 02:24:03.537087 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.537059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" event={"ID":"e627d619-379f-4409-9282-fdcc9d4ec9ef","Type":"ContainerStarted","Data":"1aaadc3f087aa78d64d4232236da55100ecc3254bc08595860e4bd562f007630"} Jul 10 02:24:03.538536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.538511 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" event={"ID":"04716cff-18b9-4372-92d0-444f4b236c51","Type":"ContainerStarted","Data":"85c71739ca6055f73090def9046d37f83361cc2e48ff99098e75fcc62564d9ab"} Jul 10 02:24:03.538620 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.538554 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" event={"ID":"04716cff-18b9-4372-92d0-444f4b236c51","Type":"ContainerStarted","Data":"ff3fbe9e6fe936096fc6a9c4c94f33410aff9d2b19bfc05eb05ac0c48691fb47"} Jul 10 02:24:03.553231 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.553189 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gqsgv" podStartSLOduration=66.54945342 podStartE2EDuration="1m10.553175938s" totalImagesPullingTime="4.003722518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:22:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:03.552411363 +0000 UTC m=+70.829200807" watchObservedRunningTime="2026-07-10 02:24:03.553175938 +0000 UTC m=+70.829965361" Jul 10 02:24:03.576052 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:03.576004 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" podStartSLOduration=2.5759855480000002 podStartE2EDuration="2.575985548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:03.574502547 +0000 UTC m=+70.851291993" watchObservedRunningTime="2026-07-10 02:24:03.575985548 +0000 UTC m=+70.852774996" Jul 10 02:24:04.542804 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:04.542763 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:06.150525 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.150488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:24:06.152699 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.152670 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlp29\" (UID: \"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0\") " pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:24:06.194040 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.194021 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pfnlm\"" Jul 10 02:24:06.202178 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.202156 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hlp29" Jul 10 02:24:06.553864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.553647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" event={"ID":"e627d619-379f-4409-9282-fdcc9d4ec9ef","Type":"ContainerStarted","Data":"a6984c60419286a0c11498aaf4e815e4da92514058ad385540525864ee2cbbb5"} Jul 10 02:24:06.571016 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:06.570988 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hlp29"] Jul 10 02:24:06.574080 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:06.574054 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc2ac12_ee8f_4492_bf5b_e0cf7f244fa0.slice/crio-dbbd92919f66dcdac2c1460f06043702b5e160cf1e153dfb0fd3975690b07598 WatchSource:0}: Error finding container dbbd92919f66dcdac2c1460f06043702b5e160cf1e153dfb0fd3975690b07598: Status 404 returned error can't find the container with id dbbd92919f66dcdac2c1460f06043702b5e160cf1e153dfb0fd3975690b07598 Jul 10 02:24:07.559359 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:07.559313 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlp29" event={"ID":"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0","Type":"ContainerStarted","Data":"dbbd92919f66dcdac2c1460f06043702b5e160cf1e153dfb0fd3975690b07598"} Jul 10 02:24:07.577454 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:07.577400 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb9468594-vbpdt" podStartSLOduration=3.415928538 podStartE2EDuration="6.57738334s" totalImagesPullingTime="3.161454802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:07.576205785 +0000 UTC m=+74.852995267" watchObservedRunningTime="2026-07-10 02:24:07.57738334 +0000 UTC m=+74.854172789" Jul 10 02:24:10.571346 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:10.571319 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlp29" event={"ID":"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0","Type":"ContainerStarted","Data":"dd9309fc87675f91fe25a51aa4668ce4eea67d12da619e8e1824d4ad2b106f29"} Jul 10 02:24:11.533645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:11.533614 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x8hz8" Jul 10 02:24:11.577702 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:11.577666 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlp29" event={"ID":"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0","Type":"ContainerStarted","Data":"adf9447f08ae892910bd27dada495bb970d66f72f8a137be6ab1f447c4acc707"} Jul 10 02:24:13.586839 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:13.586801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlp29" event={"ID":"0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0","Type":"ContainerStarted","Data":"94ea74f716881ed60f2be3cfd1a3d8dd1e6ee45915c2de526d211a926482a880"} Jul 10 02:24:13.609106 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:13.609064 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hlp29" podStartSLOduration=32.948054107 podStartE2EDuration="39.6090493s" totalImagesPullingTime="6.660995193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:23:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:13.608110059 +0000 UTC m=+80.884899524" watchObservedRunningTime="2026-07-10 02:24:13.6090493 +0000 UTC m=+80.885838746" Jul 10 02:24:14.737376 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.737341 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sdr7b"] Jul 10 02:24:14.741499 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.741479 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-78cp7"] Jul 10 02:24:14.741623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.741586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.744482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.744465 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.746248 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 02:24:14.746248 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746240 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pzdbm\"" Jul 10 02:24:14.746412 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746280 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 02:24:14.746412 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 02:24:14.746412 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746228 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 02:24:14.746412 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746400 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 02:24:14.746624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746480 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 02:24:14.746738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d22qb\"" Jul 10 02:24:14.746738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 02:24:14.747034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.746777 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 02:24:14.747034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.747002 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 02:24:14.756021 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.755999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sdr7b"] Jul 10 02:24:14.915306 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-textfile\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915466 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.915466 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915373 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-wtmp\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915466 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915466 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915452 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-root\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc6th\" (UniqueName: \"kubernetes.io/projected/b5774476-6750-4cab-b767-08748f4d4b3e-kube-api-access-nc6th\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-sys\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915801 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-metrics-client-ca\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:14.915801 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:14.915801 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:14.915653 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdr62\" (UniqueName: \"kubernetes.io/projected/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-api-access-hdr62\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016097 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-wtmp\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-root\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-wtmp\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc6th\" (UniqueName: \"kubernetes.io/projected/b5774476-6750-4cab-b767-08748f4d4b3e-kube-api-access-nc6th\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-sys\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:24:15.016465 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016488 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-metrics-client-ca\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:24:15.016542 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls podName:b5774476-6750-4cab-b767-08748f4d4b3e nodeName:}" failed. No retries permitted until 2026-07-10 02:24:15.516520237 +0000 UTC m=+82.793309676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls") pod "node-exporter-78cp7" (UID: "b5774476-6750-4cab-b767-08748f4d4b3e") : secret "node-exporter-tls" not found Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016353 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-root\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016569 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdr62\" (UniqueName: \"kubernetes.io/projected/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-api-access-hdr62\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.016623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5774476-6750-4cab-b767-08748f4d4b3e-sys\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.017222 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.016741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-textfile\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.017222 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.017061 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-textfile\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.017222 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.017112 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.017222 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.017140 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-metrics-client-ca\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.017222 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:24:15.017185 2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 02:24:15.017418 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.017238 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.017418 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.017261 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.017418 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:24:15.017286 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls podName:3e69e863-c10e-49a0-a3de-82f43c3a2b4c nodeName:}" failed. No retries permitted until 2026-07-10 02:24:15.517247728 +0000 UTC m=+82.794037166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-sdr7b" (UID: "3e69e863-c10e-49a0-a3de-82f43c3a2b4c") : secret "kube-state-metrics-tls" not found Jul 10 02:24:15.018611 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.018594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.019015 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.018999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.035075 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.035045 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdr62\" (UniqueName: \"kubernetes.io/projected/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-api-access-hdr62\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.035334 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.035317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc6th\" (UniqueName: \"kubernetes.io/projected/b5774476-6750-4cab-b767-08748f4d4b3e-kube-api-access-nc6th\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.520681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.520649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.520824 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.520688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.522896 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.522873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5774476-6750-4cab-b767-08748f4d4b3e-node-exporter-tls\") pod \"node-exporter-78cp7\" (UID: \"b5774476-6750-4cab-b767-08748f4d4b3e\") " pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.523122 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.523101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e69e863-c10e-49a0-a3de-82f43c3a2b4c-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-sdr7b\" (UID: \"3e69e863-c10e-49a0-a3de-82f43c3a2b4c\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.653390 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.653365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" Jul 10 02:24:15.659525 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.659504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78cp7" Jul 10 02:24:15.669635 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:15.669612 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5774476_6750_4cab_b767_08748f4d4b3e.slice/crio-bb54bf90612964959db0c17e8d4ee309506edfe4e77fe1d9d1d5f4bf7c9d8042 WatchSource:0}: Error finding container bb54bf90612964959db0c17e8d4ee309506edfe4e77fe1d9d1d5f4bf7c9d8042: Status 404 returned error can't find the container with id bb54bf90612964959db0c17e8d4ee309506edfe4e77fe1d9d1d5f4bf7c9d8042 Jul 10 02:24:15.783409 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:15.783385 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-sdr7b"] Jul 10 02:24:15.785605 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:15.785581 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e69e863_c10e_49a0_a3de_82f43c3a2b4c.slice/crio-54bde735081d805ebbe84905e9a28e7d350cce45aa828dc3d9560ba0e3b83eeb WatchSource:0}: Error finding container 54bde735081d805ebbe84905e9a28e7d350cce45aa828dc3d9560ba0e3b83eeb: Status 404 returned error can't find the container with id 54bde735081d805ebbe84905e9a28e7d350cce45aa828dc3d9560ba0e3b83eeb Jul 10 02:24:16.598306 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:16.598237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78cp7" event={"ID":"b5774476-6750-4cab-b767-08748f4d4b3e","Type":"ContainerStarted","Data":"bb54bf90612964959db0c17e8d4ee309506edfe4e77fe1d9d1d5f4bf7c9d8042"} Jul 10 02:24:16.599855 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:16.599789 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" event={"ID":"3e69e863-c10e-49a0-a3de-82f43c3a2b4c","Type":"ContainerStarted","Data":"54bde735081d805ebbe84905e9a28e7d350cce45aa828dc3d9560ba0e3b83eeb"} Jul 10 02:24:17.608560 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.608523 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5774476-6750-4cab-b767-08748f4d4b3e" containerID="c8db72de3bd616ee83dff4c177e2936ce2097b274a2176484188942f3cdae481" exitCode=0 Jul 10 02:24:17.608932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.608629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78cp7" event={"ID":"b5774476-6750-4cab-b767-08748f4d4b3e","Type":"ContainerDied","Data":"c8db72de3bd616ee83dff4c177e2936ce2097b274a2176484188942f3cdae481"} Jul 10 02:24:17.617438 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.617045 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" event={"ID":"3e69e863-c10e-49a0-a3de-82f43c3a2b4c","Type":"ContainerStarted","Data":"189f7d2e0dafb16b8cf7bc78b9877d5269fc62f8efb47736a4b07b2ffe5e0f51"} Jul 10 02:24:17.617438 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.617109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" event={"ID":"3e69e863-c10e-49a0-a3de-82f43c3a2b4c","Type":"ContainerStarted","Data":"41705797d986294d4093b8b084717b4a6ad1250199fdb6839dad2542f5c8a0b2"} Jul 10 02:24:17.718843 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.718764 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6fcd6bd446-dhg47"] Jul 10 02:24:17.723772 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.723752 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.730309 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730262 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 02:24:17.730426 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-csm2d\"" Jul 10 02:24:17.730489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730435 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 02:24:17.730489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e83kq793dgpmh\"" Jul 10 02:24:17.730600 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730488 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 02:24:17.730600 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730535 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 02:24:17.730704 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.730296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 02:24:17.737869 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.737845 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fcd6bd446-dhg47"] Jul 10 02:24:17.839358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839535 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839535 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839535 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3b7a0af-1717-45dc-a515-644ff7561da1-metrics-client-ca\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839535 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-grpc-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839691 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839691 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmt2d\" (UniqueName: \"kubernetes.io/projected/d3b7a0af-1717-45dc-a515-644ff7561da1-kube-api-access-zmt2d\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.839691 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.839640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940095 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3b7a0af-1717-45dc-a515-644ff7561da1-metrics-client-ca\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-grpc-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940531 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.940531 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.940308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmt2d\" (UniqueName: \"kubernetes.io/projected/d3b7a0af-1717-45dc-a515-644ff7561da1-kube-api-access-zmt2d\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.941444 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.941417 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3b7a0af-1717-45dc-a515-644ff7561da1-metrics-client-ca\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.942923 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.942864 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.943157 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.943131 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.943232 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.943210 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.943299 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.943239 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.943299 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.943252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-grpc-tls\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.943510 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.943492 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3b7a0af-1717-45dc-a515-644ff7561da1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:17.948659 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:17.948638 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmt2d\" (UniqueName: \"kubernetes.io/projected/d3b7a0af-1717-45dc-a515-644ff7561da1-kube-api-access-zmt2d\") pod \"thanos-querier-6fcd6bd446-dhg47\" (UID: \"d3b7a0af-1717-45dc-a515-644ff7561da1\") " pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:18.039955 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.039932 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:18.170980 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.170920 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6fcd6bd446-dhg47"] Jul 10 02:24:18.173366 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:18.173342 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b7a0af_1717_45dc_a515_644ff7561da1.slice/crio-067c1bbc3950333dc2b6700993e7ff04d6706b7bb95f839f44b4fca2d4a84ec7 WatchSource:0}: Error finding container 067c1bbc3950333dc2b6700993e7ff04d6706b7bb95f839f44b4fca2d4a84ec7: Status 404 returned error can't find the container with id 067c1bbc3950333dc2b6700993e7ff04d6706b7bb95f839f44b4fca2d4a84ec7 Jul 10 02:24:18.622797 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.622760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78cp7" event={"ID":"b5774476-6750-4cab-b767-08748f4d4b3e","Type":"ContainerStarted","Data":"eb3254bb7cf4bcd7b3375cc594418fe89dc423237cb94cc44ae648a8f689552c"} Jul 10 02:24:18.623227 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.622817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78cp7" event={"ID":"b5774476-6750-4cab-b767-08748f4d4b3e","Type":"ContainerStarted","Data":"89b94d0a57fc4ef0879db0287a918adf62010eb57040a4c09fd14576c00c82f8"} Jul 10 02:24:18.624784 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.624757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" event={"ID":"3e69e863-c10e-49a0-a3de-82f43c3a2b4c","Type":"ContainerStarted","Data":"f60a57d1b220e3e2f5ac0448a7dbd5e4e6a8edd164c6881aebd646ff6f235e2b"} Jul 10 02:24:18.625882 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.625857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"067c1bbc3950333dc2b6700993e7ff04d6706b7bb95f839f44b4fca2d4a84ec7"} Jul 10 02:24:18.643500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.643452 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-78cp7" podStartSLOduration=3.418579122 podStartE2EDuration="4.643437889s" totalImagesPullingTime="1.157241777s" totalInitContainerRuntime="67.61699ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:18.641875719 +0000 UTC m=+85.918665179" watchObservedRunningTime="2026-07-10 02:24:18.643437889 +0000 UTC m=+85.920227312" Jul 10 02:24:18.661890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:18.661697 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-sdr7b" podStartSLOduration=2.995874319 podStartE2EDuration="4.661681031s" totalImagesPullingTime="1.665806712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:18.661018652 +0000 UTC m=+85.937808106" watchObservedRunningTime="2026-07-10 02:24:18.661681031 +0000 UTC m=+85.938470478" Jul 10 02:24:20.636192 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:20.636157 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"191c28ee136503036f0d17ee11a8022cc50a3565bc93d11a66b958ba5b92e82c"} Jul 10 02:24:20.636645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:20.636235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"64be7461517ced4e139f17bcf2e7a1fb20172783fae7f36773e15a9274232491"} Jul 10 02:24:20.636645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:20.636304 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"5509b2b36031b3bb3903854c9b07c3bac99fbe10bbf321d24711f5bddc9788df"} Jul 10 02:24:21.646389 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:21.646311 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"81a1b7f3097e0f16af8efb57129ce6d12dd849074b08de582b5a897481c3c6a1"} Jul 10 02:24:21.646741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:21.646395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"715b147bf5f22f8b74475111b391dfea5f0c168a32b3b197f159fda8f4cd5b7c"} Jul 10 02:24:21.646741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:21.646422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" event={"ID":"d3b7a0af-1717-45dc-a515-644ff7561da1","Type":"ContainerStarted","Data":"d79befb3dbd8b7ec70dfa480e355aacb3a8ec7fb963f3e68c395861fc00d25fb"} Jul 10 02:24:21.646741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:21.646482 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:21.672941 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:21.672892 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" podStartSLOduration=1.618832278 podStartE2EDuration="4.672873735s" totalImagesPullingTime="3.054041457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:21.670480796 +0000 UTC m=+88.947270241" watchObservedRunningTime="2026-07-10 02:24:21.672873735 +0000 UTC m=+88.949663182" Jul 10 02:24:23.543477 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:23.543449 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:24.998684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:24.998656 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-l6r57"] Jul 10 02:24:25.001939 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.001924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:25.004354 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.004329 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 02:24:25.004652 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.004638 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-thjwm\"" Jul 10 02:24:25.004871 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.004858 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 02:24:25.014334 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.014316 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l6r57"] Jul 10 02:24:25.094515 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.094490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ml64\" (UniqueName: \"kubernetes.io/projected/1738b965-a6b9-472b-9daa-af498799ab65-kube-api-access-9ml64\") pod \"downloads-68c9cbb497-l6r57\" (UID: \"1738b965-a6b9-472b-9daa-af498799ab65\") " pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:25.195532 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.195503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ml64\" (UniqueName: \"kubernetes.io/projected/1738b965-a6b9-472b-9daa-af498799ab65-kube-api-access-9ml64\") pod \"downloads-68c9cbb497-l6r57\" (UID: \"1738b965-a6b9-472b-9daa-af498799ab65\") " pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:25.204916 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.204890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ml64\" (UniqueName: \"kubernetes.io/projected/1738b965-a6b9-472b-9daa-af498799ab65-kube-api-access-9ml64\") pod \"downloads-68c9cbb497-l6r57\" (UID: \"1738b965-a6b9-472b-9daa-af498799ab65\") " pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:25.310440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.310420 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:25.432775 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.432752 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-l6r57"] Jul 10 02:24:25.434897 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:25.434872 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1738b965_a6b9_472b_9daa_af498799ab65.slice/crio-b22dd3a2aa86abcd6e61d14aad3d8f8a85b251e7359222ee19b0d810f54fa548 WatchSource:0}: Error finding container b22dd3a2aa86abcd6e61d14aad3d8f8a85b251e7359222ee19b0d810f54fa548: Status 404 returned error can't find the container with id b22dd3a2aa86abcd6e61d14aad3d8f8a85b251e7359222ee19b0d810f54fa548 Jul 10 02:24:25.552004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.551983 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-749ffcb46b-cndhj" Jul 10 02:24:25.660936 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:25.660866 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l6r57" event={"ID":"1738b965-a6b9-472b-9daa-af498799ab65","Type":"ContainerStarted","Data":"b22dd3a2aa86abcd6e61d14aad3d8f8a85b251e7359222ee19b0d810f54fa548"} Jul 10 02:24:27.656711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:27.656686 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6fcd6bd446-dhg47" Jul 10 02:24:28.558036 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.557975 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" podUID="79581e9e-3894-4314-b45b-df65e181d71e" containerName="registry" containerID="cri-o://2c45bdf7739a2209ca21e8d020ffdaf847167e0a4f0e09388c12846fd4abcd76" gracePeriod=30 Jul 10 02:24:28.676120 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.676082 2560 generic.go:356] "Generic (PLEG): container finished" podID="79581e9e-3894-4314-b45b-df65e181d71e" containerID="2c45bdf7739a2209ca21e8d020ffdaf847167e0a4f0e09388c12846fd4abcd76" exitCode=0 Jul 10 02:24:28.676552 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.676124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" event={"ID":"79581e9e-3894-4314-b45b-df65e181d71e","Type":"ContainerDied","Data":"2c45bdf7739a2209ca21e8d020ffdaf847167e0a4f0e09388c12846fd4abcd76"} Jul 10 02:24:28.821176 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.821121 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:28.925694 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925665 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.925854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925716 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.925854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925749 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.925854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925787 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.925854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925822 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9lxl\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.926081 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925872 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.926081 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925904 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.926081 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.925948 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates\") pod \"79581e9e-3894-4314-b45b-df65e181d71e\" (UID: \"79581e9e-3894-4314-b45b-df65e181d71e\") " Jul 10 02:24:28.926609 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.926548 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:24:28.926718 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.926644 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:24:28.928740 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.928708 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:24:28.928843 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.928782 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:24:28.928911 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.928860 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:24:28.929176 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.929149 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl" (OuterVolumeSpecName: "kube-api-access-s9lxl") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "kube-api-access-s9lxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:24:28.929344 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.929310 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:24:28.937077 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:28.937053 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "79581e9e-3894-4314-b45b-df65e181d71e" (UID: "79581e9e-3894-4314-b45b-df65e181d71e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:24:29.027058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027027 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-bound-sa-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027056 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-installation-pull-secrets\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027067 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-registry-tls\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027076 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9lxl\" (UniqueName: \"kubernetes.io/projected/79581e9e-3894-4314-b45b-df65e181d71e-kube-api-access-s9lxl\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027085 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-trusted-ca\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027094 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/79581e9e-3894-4314-b45b-df65e181d71e-ca-trust-extracted\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027102 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/79581e9e-3894-4314-b45b-df65e181d71e-registry-certificates\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.027211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.027112 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/79581e9e-3894-4314-b45b-df65e181d71e-image-registry-private-configuration\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:24:29.681791 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.681759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" event={"ID":"79581e9e-3894-4314-b45b-df65e181d71e","Type":"ContainerDied","Data":"3db66b714608cd3a9d7913118b9c9c2abfeffd3b4d3093b698e2e962ecc904f5"} Jul 10 02:24:29.682246 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.681806 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54667b856c-j2gv6" Jul 10 02:24:29.682246 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.681818 2560 scope.go:117] "RemoveContainer" containerID="2c45bdf7739a2209ca21e8d020ffdaf847167e0a4f0e09388c12846fd4abcd76" Jul 10 02:24:29.700571 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.700550 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:24:29.705047 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:29.705023 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54667b856c-j2gv6"] Jul 10 02:24:31.224106 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:31.224023 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79581e9e-3894-4314-b45b-df65e181d71e" path="/var/lib/kubelet/pods/79581e9e-3894-4314-b45b-df65e181d71e/volumes" Jul 10 02:24:31.528622 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:31.528597 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d4b7g" Jul 10 02:24:34.945488 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.945455 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:24:34.945932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.945889 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79581e9e-3894-4314-b45b-df65e181d71e" containerName="registry" Jul 10 02:24:34.945932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.945904 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79581e9e-3894-4314-b45b-df65e181d71e" containerName="registry" Jul 10 02:24:34.946070 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.945983 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79581e9e-3894-4314-b45b-df65e181d71e" containerName="registry" Jul 10 02:24:34.975570 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.975542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:24:34.975689 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.975635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:34.978229 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.978210 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 02:24:34.979288 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.979241 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 02:24:34.979369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.979290 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 02:24:34.979369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.979354 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5l4mr\"" Jul 10 02:24:34.979369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.979362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 02:24:34.979514 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:34.979450 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 02:24:35.074005 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.073981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.074139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.074012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.074139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.074030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9sl\" (UniqueName: \"kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.074139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.074067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.074139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.074088 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.074139 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.074122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175164 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175164 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9sl\" (UniqueName: \"kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175398 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175207 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175398 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175398 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.175398 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.175353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.176132 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.176105 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.176238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.176181 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.176238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.176202 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.177995 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.177977 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.178073 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.178008 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.183694 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.183675 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9sl\" (UniqueName: \"kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl\") pod \"console-58d9548547-4llnf\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.284362 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.284331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:35.451762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.451716 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:24:35.453876 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:24:35.453841 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6526bc87_ee12_455b_9df4_f0a1322910ce.slice/crio-2119f27ed02d5968fa64c136ed42ae70b8b14ebc786cbc120ef2f651adb8ddfe WatchSource:0}: Error finding container 2119f27ed02d5968fa64c136ed42ae70b8b14ebc786cbc120ef2f651adb8ddfe: Status 404 returned error can't find the container with id 2119f27ed02d5968fa64c136ed42ae70b8b14ebc786cbc120ef2f651adb8ddfe Jul 10 02:24:35.709983 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:35.709895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9548547-4llnf" event={"ID":"6526bc87-ee12-455b-9df4-f0a1322910ce","Type":"ContainerStarted","Data":"2119f27ed02d5968fa64c136ed42ae70b8b14ebc786cbc120ef2f651adb8ddfe"} Jul 10 02:24:42.741056 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:42.741020 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-l6r57" event={"ID":"1738b965-a6b9-472b-9daa-af498799ab65","Type":"ContainerStarted","Data":"04bb666ab1c0deecb30d1d903c5fc0ba3d9f90179d419bbb12232fdeeac43846"} Jul 10 02:24:42.741679 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:42.741236 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:42.761435 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:42.761333 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-l6r57" Jul 10 02:24:42.761944 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:42.761892 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-l6r57" podStartSLOduration=1.696274352 podStartE2EDuration="18.761871189s" totalImagesPullingTime="17.065596837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:42.760465904 +0000 UTC m=+110.037255379" watchObservedRunningTime="2026-07-10 02:24:42.761871189 +0000 UTC m=+110.038660632" Jul 10 02:24:45.758081 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:45.758046 2560 generic.go:356] "Generic (PLEG): container finished" podID="f4c954d2-f958-4402-8129-3d8d188b5d1e" containerID="c81be6fcf8260adebb1c6d69c98d4b53c1d3acc9272f6cab6493b804424ce5d1" exitCode=0 Jul 10 02:24:45.758528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:45.758098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" event={"ID":"f4c954d2-f958-4402-8129-3d8d188b5d1e","Type":"ContainerDied","Data":"c81be6fcf8260adebb1c6d69c98d4b53c1d3acc9272f6cab6493b804424ce5d1"} Jul 10 02:24:45.758528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:45.758493 2560 scope.go:117] "RemoveContainer" containerID="c81be6fcf8260adebb1c6d69c98d4b53c1d3acc9272f6cab6493b804424ce5d1" Jul 10 02:24:46.764077 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:46.764036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9548547-4llnf" event={"ID":"6526bc87-ee12-455b-9df4-f0a1322910ce","Type":"ContainerStarted","Data":"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10"} Jul 10 02:24:46.766641 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:46.766611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vbp4t" event={"ID":"f4c954d2-f958-4402-8129-3d8d188b5d1e","Type":"ContainerStarted","Data":"8f73e6dd87d8f17fc9f84111244289341ea5686e2c2b92c2d19671423c0a540a"} Jul 10 02:24:46.784087 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:46.784035 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d9548547-4llnf" podStartSLOduration=2.393647852 podStartE2EDuration="12.784015491s" totalImagesPullingTime="10.390367639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:46.783464635 +0000 UTC m=+114.060254094" watchObservedRunningTime="2026-07-10 02:24:46.784015491 +0000 UTC m=+114.060804937" Jul 10 02:24:55.285092 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:55.285062 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:55.285564 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:55.285113 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:55.289833 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:55.289814 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:55.802759 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:55.802734 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:24:55.809140 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:24:55.809116 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:25:00.817824 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:00.817794 2560 generic.go:356] "Generic (PLEG): container finished" podID="16df9165-f721-462a-b576-e6c4f2bf5143" containerID="e7406304e2f976223de407c57150d4cafdaba267b4cd297a1f58efc603887f81" exitCode=0 Jul 10 02:25:00.818282 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:00.817872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" event={"ID":"16df9165-f721-462a-b576-e6c4f2bf5143","Type":"ContainerDied","Data":"e7406304e2f976223de407c57150d4cafdaba267b4cd297a1f58efc603887f81"} Jul 10 02:25:00.818282 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:00.818199 2560 scope.go:117] "RemoveContainer" containerID="e7406304e2f976223de407c57150d4cafdaba267b4cd297a1f58efc603887f81" Jul 10 02:25:01.823227 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:01.823195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jnk2g" event={"ID":"16df9165-f721-462a-b576-e6c4f2bf5143","Type":"ContainerStarted","Data":"9ad250bb0ce5fe7e962d2fb5b28dd3c560537a1fb8167016f53d63b8a3beacf7"} Jul 10 02:25:22.825059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:22.825003 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58d9548547-4llnf" podUID="6526bc87-ee12-455b-9df4-f0a1322910ce" containerName="console" containerID="cri-o://9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10" gracePeriod=15 Jul 10 02:25:23.085448 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.085387 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:25:23.251618 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251592 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.251770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251653 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.251770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251693 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.251770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251719 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq9sl\" (UniqueName: \"kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.251770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251749 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.251984 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.251797 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca\") pod \"6526bc87-ee12-455b-9df4-f0a1322910ce\" (UID: \"6526bc87-ee12-455b-9df4-f0a1322910ce\") " Jul 10 02:25:23.252133 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.252009 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:25:23.252213 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.252182 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config" (OuterVolumeSpecName: "console-config") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:25:23.252283 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.252229 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca" (OuterVolumeSpecName: "service-ca") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:25:23.253988 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.253956 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:25:23.253988 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.253967 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl" (OuterVolumeSpecName: "kube-api-access-mq9sl") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "kube-api-access-mq9sl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:25:23.254135 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.254026 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6526bc87-ee12-455b-9df4-f0a1322910ce" (UID: "6526bc87-ee12-455b-9df4-f0a1322910ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.352976 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-serving-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.352997 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6526bc87-ee12-455b-9df4-f0a1322910ce-console-oauth-config\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.353007 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq9sl\" (UniqueName: \"kubernetes.io/projected/6526bc87-ee12-455b-9df4-f0a1322910ce-kube-api-access-mq9sl\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.353015 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-console-config\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.353023 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-service-ca\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.353028 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.353030 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6526bc87-ee12-455b-9df4-f0a1322910ce-oauth-serving-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:25:23.906642 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.906607 2560 generic.go:356] "Generic (PLEG): container finished" podID="6526bc87-ee12-455b-9df4-f0a1322910ce" containerID="9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10" exitCode=0 Jul 10 02:25:23.907078 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.906647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9548547-4llnf" event={"ID":"6526bc87-ee12-455b-9df4-f0a1322910ce","Type":"ContainerDied","Data":"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10"} Jul 10 02:25:23.907078 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.906656 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d9548547-4llnf" Jul 10 02:25:23.907078 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.906693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d9548547-4llnf" event={"ID":"6526bc87-ee12-455b-9df4-f0a1322910ce","Type":"ContainerDied","Data":"2119f27ed02d5968fa64c136ed42ae70b8b14ebc786cbc120ef2f651adb8ddfe"} Jul 10 02:25:23.907078 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.906721 2560 scope.go:117] "RemoveContainer" containerID="9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10" Jul 10 02:25:23.915405 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.915389 2560 scope.go:117] "RemoveContainer" containerID="9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10" Jul 10 02:25:23.915638 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:25:23.915618 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10\": container with ID starting with 9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10 not found: ID does not exist" containerID="9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10" Jul 10 02:25:23.915694 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.915647 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10"} err="failed to get container status \"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10\": rpc error: code = NotFound desc = could not find container \"9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10\": container with ID starting with 9ad8fb04fb0c947bf8e3df64ef9fcf5b7122c8b053838c9a1f1399dc16c14d10 not found: ID does not exist" Jul 10 02:25:23.930146 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.930120 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:25:23.937105 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:23.937082 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d9548547-4llnf"] Jul 10 02:25:25.222683 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:25.222645 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6526bc87-ee12-455b-9df4-f0a1322910ce" path="/var/lib/kubelet/pods/6526bc87-ee12-455b-9df4-f0a1322910ce/volumes" Jul 10 02:25:32.227665 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.227634 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:25:32.228058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.227902 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6526bc87-ee12-455b-9df4-f0a1322910ce" containerName="console" Jul 10 02:25:32.228058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.227913 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6526bc87-ee12-455b-9df4-f0a1322910ce" containerName="console" Jul 10 02:25:32.228058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.227989 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6526bc87-ee12-455b-9df4-f0a1322910ce" containerName="console" Jul 10 02:25:32.233912 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.233884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.236672 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.236649 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5l4mr\"" Jul 10 02:25:32.237601 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.237579 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 02:25:32.237710 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.237603 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 02:25:32.237710 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.237596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 02:25:32.237710 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.237596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 02:25:32.237710 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.237647 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 02:25:32.243129 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.243114 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 02:25:32.276362 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.276344 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:25:32.414223 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414193 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414251 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414310 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.414520 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.414376 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbnzl\" (UniqueName: \"kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515095 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515439 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515439 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbnzl\" (UniqueName: \"kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.515921 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515898 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.516020 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515942 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.516020 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.515990 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.516135 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.516028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.517630 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.517602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.517729 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.517635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.524678 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.524657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbnzl\" (UniqueName: \"kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl\") pod \"console-6d8465cb5b-j6dw8\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.544233 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.544213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:32.674918 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.674894 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:25:32.676752 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:25:32.676715 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8406e5fd_7202_46ba_838c_5df7a22736ab.slice/crio-2c40d295018404cb5d3569a8f6b4c70c3028c7d7af58256056fa71060e299c0a WatchSource:0}: Error finding container 2c40d295018404cb5d3569a8f6b4c70c3028c7d7af58256056fa71060e299c0a: Status 404 returned error can't find the container with id 2c40d295018404cb5d3569a8f6b4c70c3028c7d7af58256056fa71060e299c0a Jul 10 02:25:32.940893 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.940802 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8465cb5b-j6dw8" event={"ID":"8406e5fd-7202-46ba-838c-5df7a22736ab","Type":"ContainerStarted","Data":"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153"} Jul 10 02:25:32.940893 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.940851 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8465cb5b-j6dw8" event={"ID":"8406e5fd-7202-46ba-838c-5df7a22736ab","Type":"ContainerStarted","Data":"2c40d295018404cb5d3569a8f6b4c70c3028c7d7af58256056fa71060e299c0a"} Jul 10 02:25:32.960764 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:32.960721 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8465cb5b-j6dw8" podStartSLOduration=0.960706491 podStartE2EDuration="960.706491ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:25:32.959666268 +0000 UTC m=+160.236455714" watchObservedRunningTime="2026-07-10 02:25:32.960706491 +0000 UTC m=+160.237495994" Jul 10 02:25:42.544829 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:42.544794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:42.545252 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:42.544849 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:42.549503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:42.549483 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:25:42.982250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:25:42.982177 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:26:44.526558 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.526527 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ffcn2"] Jul 10 02:26:44.528447 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.528430 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.531116 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.531100 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-867pm\"" Jul 10 02:26:44.532136 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.532115 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 02:26:44.532368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.532351 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 02:26:44.543356 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.543334 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ffcn2"] Jul 10 02:26:44.600756 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.600733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.600846 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.600777 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bc7v\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-kube-api-access-7bc7v\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.702045 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.702021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.702128 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.702065 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bc7v\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-kube-api-access-7bc7v\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.711339 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.711317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.711617 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.711597 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bc7v\" (UniqueName: \"kubernetes.io/projected/d1819a33-0aed-47ca-a576-dc8e8cf9dc9c-kube-api-access-7bc7v\") pod \"cert-manager-webhook-6486ffd997-ffcn2\" (UID: \"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.837554 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.837499 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:44.968620 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:44.968594 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ffcn2"] Jul 10 02:26:44.972109 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:26:44.972073 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1819a33_0aed_47ca_a576_dc8e8cf9dc9c.slice/crio-449ec0d4df6f9a74c6c0f1c63a669d9d3cb377c6fca24b4a56d1c2c57ef501d1 WatchSource:0}: Error finding container 449ec0d4df6f9a74c6c0f1c63a669d9d3cb377c6fca24b4a56d1c2c57ef501d1: Status 404 returned error can't find the container with id 449ec0d4df6f9a74c6c0f1c63a669d9d3cb377c6fca24b4a56d1c2c57ef501d1 Jul 10 02:26:45.200719 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:45.200656 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" event={"ID":"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c","Type":"ContainerStarted","Data":"449ec0d4df6f9a74c6c0f1c63a669d9d3cb377c6fca24b4a56d1c2c57ef501d1"} Jul 10 02:26:48.214327 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:48.214232 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" event={"ID":"d1819a33-0aed-47ca-a576-dc8e8cf9dc9c","Type":"ContainerStarted","Data":"59d0f4c9b6ab0eddc435c8cd7b8642592c4686ad456577922281ff82556b68fa"} Jul 10 02:26:48.214327 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:48.214307 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:48.233404 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:48.233360 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" podStartSLOduration=1.327614154 podStartE2EDuration="4.233344951s" totalImagesPullingTime="2.905730797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:26:48.231415277 +0000 UTC m=+235.508204723" watchObservedRunningTime="2026-07-10 02:26:48.233344951 +0000 UTC m=+235.510134397" Jul 10 02:26:54.220294 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:54.220243 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ffcn2" Jul 10 02:26:56.058847 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.058813 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v52t"] Jul 10 02:26:56.060851 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.060834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.063777 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.063755 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z4s6f\"" Jul 10 02:26:56.073342 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.073320 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v52t"] Jul 10 02:26:56.189778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.189752 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlmz\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-kube-api-access-rwlmz\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.189915 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.189808 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.290520 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.290492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.290676 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.290534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlmz\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-kube-api-access-rwlmz\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.305865 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.305842 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlmz\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-kube-api-access-rwlmz\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.311472 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.311410 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5343dd33-6d6b-4bbf-905e-743e5622e57e-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v52t\" (UID: \"5343dd33-6d6b-4bbf-905e-743e5622e57e\") " pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.369460 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.369438 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6v52t" Jul 10 02:26:56.491880 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:56.491711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v52t"] Jul 10 02:26:56.493854 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:26:56.493821 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5343dd33_6d6b_4bbf_905e_743e5622e57e.slice/crio-a8a4bbc201620e7a1c03a81bf5c6e8ab22e361c8967bde768e3983369e6e3f49 WatchSource:0}: Error finding container a8a4bbc201620e7a1c03a81bf5c6e8ab22e361c8967bde768e3983369e6e3f49: Status 404 returned error can't find the container with id a8a4bbc201620e7a1c03a81bf5c6e8ab22e361c8967bde768e3983369e6e3f49 Jul 10 02:26:57.244993 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:57.244962 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6v52t" event={"ID":"5343dd33-6d6b-4bbf-905e-743e5622e57e","Type":"ContainerStarted","Data":"5319c5af900f27155147c87fe81ac4dae672dd3e461956c7b0d28e4dbc8f5561"} Jul 10 02:26:57.244993 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:57.245000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6v52t" event={"ID":"5343dd33-6d6b-4bbf-905e-743e5622e57e","Type":"ContainerStarted","Data":"a8a4bbc201620e7a1c03a81bf5c6e8ab22e361c8967bde768e3983369e6e3f49"} Jul 10 02:26:57.263910 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:26:57.263864 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6v52t" podStartSLOduration=1.2638502360000001 podStartE2EDuration="1.263850236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:26:57.263110644 +0000 UTC m=+244.539900102" watchObservedRunningTime="2026-07-10 02:26:57.263850236 +0000 UTC m=+244.540639681" Jul 10 02:27:17.949397 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.949243 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t"] Jul 10 02:27:17.952536 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.952519 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:17.956398 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.956376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 02:27:17.956663 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.956640 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 02:27:17.956755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.956686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 02:27:17.956755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.956744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j45p7\"" Jul 10 02:27:17.957939 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.957918 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 02:27:17.984221 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:17.984200 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t"] Jul 10 02:27:18.045616 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.045589 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.045767 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.045644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-tmp\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.045767 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.045669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtpp4\" (UniqueName: \"kubernetes.io/projected/72cbaf8e-0475-476e-b827-19213ced8b66-kube-api-access-qtpp4\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.045767 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.045691 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-manifests\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.045767 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.045711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.146629 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.146596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-tmp\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.146629 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.146633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtpp4\" (UniqueName: \"kubernetes.io/projected/72cbaf8e-0475-476e-b827-19213ced8b66-kube-api-access-qtpp4\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.146842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.146665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-manifests\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.146842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.146696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.146842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.146739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.147410 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.147387 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-manifests\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.148966 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.148936 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72cbaf8e-0475-476e-b827-19213ced8b66-tmp\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.149189 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.149172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.149303 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.149286 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/72cbaf8e-0475-476e-b827-19213ced8b66-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.185793 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.185768 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtpp4\" (UniqueName: \"kubernetes.io/projected/72cbaf8e-0475-476e-b827-19213ced8b66-kube-api-access-qtpp4\") pod \"opendatahub-operator-controller-manager-7fc46458b-plz8t\" (UID: \"72cbaf8e-0475-476e-b827-19213ced8b66\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.263027 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.263009 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:18.326650 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.325056 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm"] Jul 10 02:27:18.330649 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.328919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.339049 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.339305 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.339486 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.339659 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rp2w9\"" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.339880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 02:27:18.343315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.340098 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:27:18.346167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.346105 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm"] Jul 10 02:27:18.425849 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.425817 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t"] Jul 10 02:27:18.429187 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:27:18.429159 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72cbaf8e_0475_476e_b827_19213ced8b66.slice/crio-c56fb4ab40699d2132ea22ecd73aac7e76c757551eeb774b09eb3e47a30c2151 WatchSource:0}: Error finding container c56fb4ab40699d2132ea22ecd73aac7e76c757551eeb774b09eb3e47a30c2151: Status 404 returned error can't find the container with id c56fb4ab40699d2132ea22ecd73aac7e76c757551eeb774b09eb3e47a30c2151 Jul 10 02:27:18.451777 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.451755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.451873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.451785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-metrics-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.451873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.451824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jx4k\" (UniqueName: \"kubernetes.io/projected/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-kube-api-access-2jx4k\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.451959 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.451874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-manager-config\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.553141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.553064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jx4k\" (UniqueName: \"kubernetes.io/projected/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-kube-api-access-2jx4k\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.553141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.553105 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-manager-config\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.553141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.553137 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.553373 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.553154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-metrics-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.553832 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.553809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-manager-config\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.555661 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.555640 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.555736 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.555644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-metrics-cert\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.563829 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.563808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jx4k\" (UniqueName: \"kubernetes.io/projected/9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd-kube-api-access-2jx4k\") pod \"lws-controller-manager-5fcbd9fc8-sglnm\" (UID: \"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd\") " pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.657466 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.657444 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:18.794022 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:18.793966 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm"] Jul 10 02:27:18.796009 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:27:18.795984 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8d8706_d8ca_4e0f_89c4_b926a19ea2dd.slice/crio-f1082e0b5ec9b5d26bf2caec8d8696a9d26dce47755da67ef6ff060531268d13 WatchSource:0}: Error finding container f1082e0b5ec9b5d26bf2caec8d8696a9d26dce47755da67ef6ff060531268d13: Status 404 returned error can't find the container with id f1082e0b5ec9b5d26bf2caec8d8696a9d26dce47755da67ef6ff060531268d13 Jul 10 02:27:19.335487 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:19.335447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" event={"ID":"72cbaf8e-0475-476e-b827-19213ced8b66","Type":"ContainerStarted","Data":"c56fb4ab40699d2132ea22ecd73aac7e76c757551eeb774b09eb3e47a30c2151"} Jul 10 02:27:19.336782 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:19.336750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" event={"ID":"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd","Type":"ContainerStarted","Data":"f1082e0b5ec9b5d26bf2caec8d8696a9d26dce47755da67ef6ff060531268d13"} Jul 10 02:27:23.356771 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:23.356735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" event={"ID":"9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd","Type":"ContainerStarted","Data":"172292c025e4cdf0ba2cba6d669e792e728a38c60613bdbca3311281520ba272"} Jul 10 02:27:23.357208 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:23.356865 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:23.358381 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:23.358354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" event={"ID":"72cbaf8e-0475-476e-b827-19213ced8b66","Type":"ContainerStarted","Data":"3a9703978f37a739fa09ce79f2dca1e5585c0eda35d5fe20d36c0b386b56d4aa"} Jul 10 02:27:23.377351 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:23.377309 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" podStartSLOduration=1.455057324 podStartE2EDuration="5.377290895s" totalImagesPullingTime="3.922233571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:27:23.376651141 +0000 UTC m=+270.653440608" watchObservedRunningTime="2026-07-10 02:27:23.377290895 +0000 UTC m=+270.654080339" Jul 10 02:27:24.364148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:24.364114 2560 generic.go:356] "Generic (PLEG): container finished" podID="72cbaf8e-0475-476e-b827-19213ced8b66" containerID="3a9703978f37a739fa09ce79f2dca1e5585c0eda35d5fe20d36c0b386b56d4aa" exitCode=0 Jul 10 02:27:24.364527 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:24.364178 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" event={"ID":"72cbaf8e-0475-476e-b827-19213ced8b66","Type":"ContainerDied","Data":"3a9703978f37a739fa09ce79f2dca1e5585c0eda35d5fe20d36c0b386b56d4aa"} Jul 10 02:27:25.370640 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:25.370604 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" event={"ID":"72cbaf8e-0475-476e-b827-19213ced8b66","Type":"ContainerStarted","Data":"f7bfea9a1093413041df7688338ff346723362f4428b8607c3d2c2213792c286"} Jul 10 02:27:25.371103 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:25.370729 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:25.397141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:25.397079 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" podStartSLOduration=2.646508317 podStartE2EDuration="8.397060803s" totalImagesPullingTime="4.689233261s" totalInitContainerRuntime="1.061319225s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:27:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:27:25.395466892 +0000 UTC m=+272.672256358" watchObservedRunningTime="2026-07-10 02:27:25.397060803 +0000 UTC m=+272.673850246" Jul 10 02:27:34.366580 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:34.366551 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5fcbd9fc8-sglnm" Jul 10 02:27:36.376257 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:36.376231 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc46458b-plz8t" Jul 10 02:27:47.479955 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.479926 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5"] Jul 10 02:27:47.482998 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.482977 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.487778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.487755 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 02:27:47.487778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.487771 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 02:27:47.487925 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.487910 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 02:27:47.489129 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.489108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w22kx\"" Jul 10 02:27:47.515188 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.515165 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5"] Jul 10 02:27:47.584175 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584149 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584330 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584179 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584330 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584330 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584483 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584587 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584488 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5fds\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-kube-api-access-k5fds\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.584587 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.584538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685256 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5fds\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-kube-api-access-k5fds\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685381 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685443 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685755 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.685907 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.685780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.686238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.686215 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.686361 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.686217 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.686361 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.686350 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.686527 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.686501 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.688112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.688087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.688258 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.688236 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.700116 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.700096 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.721403 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.721380 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5fds\" (UniqueName: \"kubernetes.io/projected/fdb70edb-29b0-4e9b-a083-cac4dd20f71e-kube-api-access-k5fds\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qv7m5\" (UID: \"fdb70edb-29b0-4e9b-a083-cac4dd20f71e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.792684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.792666 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:47.962381 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:47.962356 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5"] Jul 10 02:27:47.964842 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:27:47.964814 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb70edb_29b0_4e9b_a083_cac4dd20f71e.slice/crio-43bf7bc4caa8ece81f94b0f2b2dc9af1656baf57781ecdcae1d9087bbd8c8703 WatchSource:0}: Error finding container 43bf7bc4caa8ece81f94b0f2b2dc9af1656baf57781ecdcae1d9087bbd8c8703: Status 404 returned error can't find the container with id 43bf7bc4caa8ece81f94b0f2b2dc9af1656baf57781ecdcae1d9087bbd8c8703 Jul 10 02:27:48.160661 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.160584 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4"] Jul 10 02:27:48.164632 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.164615 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.167162 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.167144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 02:27:48.167287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.167143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 02:27:48.167287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.167174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c6md5\"" Jul 10 02:27:48.167579 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.167567 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 02:27:48.180211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.180190 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4"] Jul 10 02:27:48.189626 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189604 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189737 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189642 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdmlw\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-kube-api-access-rdmlw\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189737 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189853 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189853 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/034a48a0-d150-4fb0-83f1-958973e7d240-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189958 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.189958 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.189904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290560 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290572 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdmlw\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-kube-api-access-rdmlw\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290846 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290846 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/034a48a0-d150-4fb0-83f1-958973e7d240-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.290846 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.290762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.291448 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.291402 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.292965 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.292945 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/034a48a0-d150-4fb0-83f1-958973e7d240-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.293027 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.292998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.293322 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.293307 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.293369 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.293351 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/034a48a0-d150-4fb0-83f1-958973e7d240-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.300811 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.300789 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.300891 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.300840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdmlw\" (UniqueName: \"kubernetes.io/projected/034a48a0-d150-4fb0-83f1-958973e7d240-kube-api-access-rdmlw\") pod \"istiod-openshift-gateway-94bb8fbfd-gxvz4\" (UID: \"034a48a0-d150-4fb0-83f1-958973e7d240\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.463765 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.463698 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" event={"ID":"fdb70edb-29b0-4e9b-a083-cac4dd20f71e","Type":"ContainerStarted","Data":"43bf7bc4caa8ece81f94b0f2b2dc9af1656baf57781ecdcae1d9087bbd8c8703"} Jul 10 02:27:48.473962 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.473944 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:48.634649 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:48.634615 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4"] Jul 10 02:27:48.637623 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:27:48.637595 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034a48a0_d150_4fb0_83f1_958973e7d240.slice/crio-347515e052520087a53f67c260cbd040f2063aa443f61c5697c4544710f25b3a WatchSource:0}: Error finding container 347515e052520087a53f67c260cbd040f2063aa443f61c5697c4544710f25b3a: Status 404 returned error can't find the container with id 347515e052520087a53f67c260cbd040f2063aa443f61c5697c4544710f25b3a Jul 10 02:27:49.471805 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:49.471765 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" event={"ID":"034a48a0-d150-4fb0-83f1-958973e7d240","Type":"ContainerStarted","Data":"347515e052520087a53f67c260cbd040f2063aa443f61c5697c4544710f25b3a"} Jul 10 02:27:51.793920 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:51.793887 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:27:51.794163 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:51.793958 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:27:51.794163 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:51.793989 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:27:51.794922 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:51.794899 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:27:51.794958 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:51.794947 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:27:52.491125 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.491093 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" event={"ID":"fdb70edb-29b0-4e9b-a083-cac4dd20f71e","Type":"ContainerStarted","Data":"d9cafef0e7abcf832077b85276e401ba164b2febd476595fa998616782799592"} Jul 10 02:27:52.492633 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.492600 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" event={"ID":"034a48a0-d150-4fb0-83f1-958973e7d240","Type":"ContainerStarted","Data":"3bdd99ed597926452ef52c56ea88fb0e624ec705ff5ae68ea9102504d1d97e83"} Jul 10 02:27:52.492798 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.492783 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:52.494226 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.494191 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-gxvz4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 02:27:52.494341 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.494248 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" podUID="034a48a0-d150-4fb0-83f1-958973e7d240" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 02:27:52.518465 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.518425 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" podStartSLOduration=1.691605401 podStartE2EDuration="5.518412219s" totalImagesPullingTime="3.826806818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:27:52.516340022 +0000 UTC m=+299.793129489" watchObservedRunningTime="2026-07-10 02:27:52.518412219 +0000 UTC m=+299.795201664" Jul 10 02:27:52.556844 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.556801 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" podStartSLOduration=1.401730535 podStartE2EDuration="4.556788764s" totalImagesPullingTime="3.155058229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:27:52.552998938 +0000 UTC m=+299.829788385" watchObservedRunningTime="2026-07-10 02:27:52.556788764 +0000 UTC m=+299.833578210" Jul 10 02:27:52.793732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.793701 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:52.797997 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:52.797975 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:53.088499 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.088426 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:27:53.089492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.089467 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:27:53.094903 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.094887 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 02:27:53.497041 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.496907 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:27:53.513657 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.497401 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gxvz4" Jul 10 02:27:53.513657 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:27:53.497884 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qv7m5" Jul 10 02:30:02.387589 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.387550 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h"] Jul 10 02:30:02.391077 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.391056 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.393812 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.393791 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-96g64\"" Jul 10 02:30:02.409141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.409120 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h"] Jul 10 02:30:02.476884 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.476846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18920e3-8734-471d-b4db-a16129a305be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477048 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.476935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477048 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.476965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477048 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.476994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18920e3-8734-471d-b4db-a16129a305be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477204 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.477054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477204 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.477136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.477212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.477249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb6n6\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-kube-api-access-jb6n6\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.477411 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.477348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.577998 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.577959 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.577998 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578023 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb6n6\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-kube-api-access-jb6n6\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18920e3-8734-471d-b4db-a16129a305be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18920e3-8734-471d-b4db-a16129a305be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578600 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578443 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578763 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578739 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578833 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.578833 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578798 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.579015 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.578991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18920e3-8734-471d-b4db-a16129a305be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.580530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.580510 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18920e3-8734-471d-b4db-a16129a305be-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.580669 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.580650 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18920e3-8734-471d-b4db-a16129a305be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.587365 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.587342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.587475 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.587419 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb6n6\" (UniqueName: \"kubernetes.io/projected/b18920e3-8734-471d-b4db-a16129a305be-kube-api-access-jb6n6\") pod \"maas-default-gateway-openshift-default-56654dcf7-sw79h\" (UID: \"b18920e3-8734-471d-b4db-a16129a305be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.700854 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.700785 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:02.834946 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.834923 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h"] Jul 10 02:30:02.836866 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:30:02.836841 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18920e3_8734_471d_b4db_a16129a305be.slice/crio-c098e9451148d3cffb98d9ddfdec455d14dcf9239c94a49f55f4c0020802f47d WatchSource:0}: Error finding container c098e9451148d3cffb98d9ddfdec455d14dcf9239c94a49f55f4c0020802f47d: Status 404 returned error can't find the container with id c098e9451148d3cffb98d9ddfdec455d14dcf9239c94a49f55f4c0020802f47d Jul 10 02:30:02.838823 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.838796 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:30:02.838890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.838865 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:30:02.838925 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:02.838897 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:30:03.045523 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.045485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" event={"ID":"b18920e3-8734-471d-b4db-a16129a305be","Type":"ContainerStarted","Data":"9c93852fd5c86a8ec7495281d77073b9b545fa3980e9c3b8f58434ba0af98d9f"} Jul 10 02:30:03.045723 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.045536 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" event={"ID":"b18920e3-8734-471d-b4db-a16129a305be","Type":"ContainerStarted","Data":"c098e9451148d3cffb98d9ddfdec455d14dcf9239c94a49f55f4c0020802f47d"} Jul 10 02:30:03.067309 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.067230 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podStartSLOduration=1.067207936 podStartE2EDuration="1.067207936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:30:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:30:03.064998499 +0000 UTC m=+430.341787973" watchObservedRunningTime="2026-07-10 02:30:03.067207936 +0000 UTC m=+430.343997382" Jul 10 02:30:03.277503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.277476 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:30:03.701825 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.701783 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:03.706305 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:03.706282 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:04.050059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:04.050030 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:04.050907 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:04.050888 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:30:28.297469 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.297410 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d8465cb5b-j6dw8" podUID="8406e5fd-7202-46ba-838c-5df7a22736ab" containerName="console" containerID="cri-o://b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153" gracePeriod=15 Jul 10 02:30:28.543181 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.543156 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:30:28.671645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671558 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.671645 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671642 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.671859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671674 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.671859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671721 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbnzl\" (UniqueName: \"kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.671859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671773 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.672011 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671926 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.672011 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.671984 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert\") pod \"8406e5fd-7202-46ba-838c-5df7a22736ab\" (UID: \"8406e5fd-7202-46ba-838c-5df7a22736ab\") " Jul 10 02:30:28.672130 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672093 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:30:28.672255 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672128 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca" (OuterVolumeSpecName: "service-ca") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:30:28.672450 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672429 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-trusted-ca-bundle\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.672514 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672440 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:30:28.672514 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672456 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-service-ca\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.672914 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.672888 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config" (OuterVolumeSpecName: "console-config") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:30:28.674347 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.674320 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl" (OuterVolumeSpecName: "kube-api-access-sbnzl") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "kube-api-access-sbnzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:30:28.674449 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.674344 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:30:28.674449 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.674363 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8406e5fd-7202-46ba-838c-5df7a22736ab" (UID: "8406e5fd-7202-46ba-838c-5df7a22736ab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:30:28.772983 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.772966 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-oauth-serving-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.772983 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.772985 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-serving-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.773101 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.772994 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406e5fd-7202-46ba-838c-5df7a22736ab-console-oauth-config\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.773101 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.773002 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406e5fd-7202-46ba-838c-5df7a22736ab-console-config\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:28.773101 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:28.773011 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbnzl\" (UniqueName: \"kubernetes.io/projected/8406e5fd-7202-46ba-838c-5df7a22736ab-kube-api-access-sbnzl\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:30:29.155778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.155743 2560 generic.go:356] "Generic (PLEG): container finished" podID="8406e5fd-7202-46ba-838c-5df7a22736ab" containerID="b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153" exitCode=0 Jul 10 02:30:29.155938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.155785 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8465cb5b-j6dw8" Jul 10 02:30:29.155938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.155826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8465cb5b-j6dw8" event={"ID":"8406e5fd-7202-46ba-838c-5df7a22736ab","Type":"ContainerDied","Data":"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153"} Jul 10 02:30:29.155938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.155879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8465cb5b-j6dw8" event={"ID":"8406e5fd-7202-46ba-838c-5df7a22736ab","Type":"ContainerDied","Data":"2c40d295018404cb5d3569a8f6b4c70c3028c7d7af58256056fa71060e299c0a"} Jul 10 02:30:29.155938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.155906 2560 scope.go:117] "RemoveContainer" containerID="b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153" Jul 10 02:30:29.164462 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.164446 2560 scope.go:117] "RemoveContainer" containerID="b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153" Jul 10 02:30:29.164722 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:30:29.164703 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153\": container with ID starting with b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153 not found: ID does not exist" containerID="b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153" Jul 10 02:30:29.164761 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.164729 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153"} err="failed to get container status \"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153\": rpc error: code = NotFound desc = could not find container \"b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153\": container with ID starting with b043a991357f76b15f4824c5ed6aee3100e8f468b3c8043158ba39a74c8bd153 not found: ID does not exist" Jul 10 02:30:29.178699 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.178678 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:30:29.185024 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.185005 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d8465cb5b-j6dw8"] Jul 10 02:30:29.228190 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:30:29.228168 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8406e5fd-7202-46ba-838c-5df7a22736ab" path="/var/lib/kubelet/pods/8406e5fd-7202-46ba-838c-5df7a22736ab/volumes" Jul 10 02:31:20.668970 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.668938 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-25htw"] Jul 10 02:31:20.669386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.669243 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8406e5fd-7202-46ba-838c-5df7a22736ab" containerName="console" Jul 10 02:31:20.669386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.669253 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8406e5fd-7202-46ba-838c-5df7a22736ab" containerName="console" Jul 10 02:31:20.669386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.669340 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8406e5fd-7202-46ba-838c-5df7a22736ab" containerName="console" Jul 10 02:31:20.672215 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.672197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.674835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.674813 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cj89r\"" Jul 10 02:31:20.674984 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.674925 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 02:31:20.687984 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.687962 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-25htw"] Jul 10 02:31:20.766833 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.766801 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbjpl\" (UniqueName: \"kubernetes.io/projected/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-kube-api-access-dbjpl\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.766985 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.766857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-plugins-config-volume\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.868189 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.868162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-plugins-config-volume\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.868340 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.868247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbjpl\" (UniqueName: \"kubernetes.io/projected/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-kube-api-access-dbjpl\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.868887 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.868868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-plugins-config-volume\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.882283 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.882249 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbjpl\" (UniqueName: \"kubernetes.io/projected/6c32bd56-9cac-45c8-9a58-9b96b36e19e6-kube-api-access-dbjpl\") pod \"payload-processing-58f86c7798-25htw\" (UID: \"6c32bd56-9cac-45c8-9a58-9b96b36e19e6\") " pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:20.983008 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:20.982945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:21.131616 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:21.131593 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-25htw"] Jul 10 02:31:21.133576 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:31:21.133537 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c32bd56_9cac_45c8_9a58_9b96b36e19e6.slice/crio-af26040a4cbd18a6ee68eaedba381ad082e8f744e0bc477828bc8c4745bc6ad2 WatchSource:0}: Error finding container af26040a4cbd18a6ee68eaedba381ad082e8f744e0bc477828bc8c4745bc6ad2: Status 404 returned error can't find the container with id af26040a4cbd18a6ee68eaedba381ad082e8f744e0bc477828bc8c4745bc6ad2 Jul 10 02:31:21.135575 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:21.135560 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:31:21.367848 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:21.367822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-25htw" event={"ID":"6c32bd56-9cac-45c8-9a58-9b96b36e19e6","Type":"ContainerStarted","Data":"af26040a4cbd18a6ee68eaedba381ad082e8f744e0bc477828bc8c4745bc6ad2"} Jul 10 02:31:23.377389 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:23.377355 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-25htw" event={"ID":"6c32bd56-9cac-45c8-9a58-9b96b36e19e6","Type":"ContainerStarted","Data":"3e931e426c372844e273ab1ae536ae4cd9d8ce5492f163802c6d886ddd930ea8"} Jul 10 02:31:23.377842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:23.377459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:23.413542 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:23.413498 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-25htw" podStartSLOduration=2.0125877660000002 podStartE2EDuration="3.413481806s" totalImagesPullingTime="1.40089404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:31:23.412911783 +0000 UTC m=+510.689701231" watchObservedRunningTime="2026-07-10 02:31:23.413481806 +0000 UTC m=+510.690271252" Jul 10 02:31:30.410488 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.410455 2560 generic.go:356] "Generic (PLEG): container finished" podID="b18920e3-8734-471d-b4db-a16129a305be" containerID="9c93852fd5c86a8ec7495281d77073b9b545fa3980e9c3b8f58434ba0af98d9f" exitCode=0 Jul 10 02:31:30.410905 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.410534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" event={"ID":"b18920e3-8734-471d-b4db-a16129a305be","Type":"ContainerDied","Data":"9c93852fd5c86a8ec7495281d77073b9b545fa3980e9c3b8f58434ba0af98d9f"} Jul 10 02:31:30.410974 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.410907 2560 scope.go:117] "RemoveContainer" containerID="9c93852fd5c86a8ec7495281d77073b9b545fa3980e9c3b8f58434ba0af98d9f" Jul 10 02:31:30.411875 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.411853 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:31:30.411942 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.411920 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:31:30.411980 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:30.411955 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:31:31.417338 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:31.417300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" event={"ID":"b18920e3-8734-471d-b4db-a16129a305be","Type":"ContainerStarted","Data":"b2e9e524d2362a2e62d005f0362870891ea11c68a89e9de80cfb490dfe73eb5b"} Jul 10 02:31:31.701860 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:31.701783 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:31:31.702510 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:31.702485 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sw79h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 10 02:31:31.702559 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:31.702543 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podUID="b18920e3-8734-471d-b4db-a16129a305be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 10 02:31:32.701234 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:32.701196 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sw79h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 10 02:31:32.701629 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:32.701285 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podUID="b18920e3-8734-471d-b4db-a16129a305be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 10 02:31:33.701652 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:33.701618 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sw79h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 10 02:31:33.702120 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:33.701677 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podUID="b18920e3-8734-471d-b4db-a16129a305be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 10 02:31:34.050376 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:34.050343 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:31:34.383620 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:34.383538 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-25htw" Jul 10 02:31:34.702051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:34.701969 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sw79h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 10 02:31:34.702051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:34.702022 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podUID="b18920e3-8734-471d-b4db-a16129a305be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 10 02:31:35.702107 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:35.702069 2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-sw79h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 10 02:31:35.702523 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:35.702132 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" podUID="b18920e3-8734-471d-b4db-a16129a305be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 10 02:31:36.704935 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:36.704907 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:31:36.705948 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:36.705928 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-sw79h" Jul 10 02:31:55.336428 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.336340 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz"] Jul 10 02:31:55.339675 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.339637 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342319 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7rw8\" (UniqueName: \"kubernetes.io/projected/fe765ea0-799d-47c2-bd14-4b0327dfb386-kube-api-access-k7rw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342461 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe765ea0-799d-47c2-bd14-4b0327dfb386-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342461 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ggkvw\"" Jul 10 02:31:55.342461 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342461 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342445 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 02:31:55.342708 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342708 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.342708 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.342600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.343296 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.343250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 02:31:55.343400 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.343373 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 02:31:55.354086 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.354065 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz"] Jul 10 02:31:55.443342 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7rw8\" (UniqueName: \"kubernetes.io/projected/fe765ea0-799d-47c2-bd14-4b0327dfb386-kube-api-access-k7rw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe765ea0-799d-47c2-bd14-4b0327dfb386-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443492 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443780 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443824 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.443900 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.443876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.445534 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.445518 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe765ea0-799d-47c2-bd14-4b0327dfb386-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.445805 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.445788 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe765ea0-799d-47c2-bd14-4b0327dfb386-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.452306 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.452286 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7rw8\" (UniqueName: \"kubernetes.io/projected/fe765ea0-799d-47c2-bd14-4b0327dfb386-kube-api-access-k7rw8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z6qxz\" (UID: \"fe765ea0-799d-47c2-bd14-4b0327dfb386\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.650138 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.650066 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:31:55.787582 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:55.787557 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz"] Jul 10 02:31:55.789521 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:31:55.789493 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe765ea0_799d_47c2_bd14_4b0327dfb386.slice/crio-73d132a7233c44110712bdf3766d026892346b93c74827c02db940931e390442 WatchSource:0}: Error finding container 73d132a7233c44110712bdf3766d026892346b93c74827c02db940931e390442: Status 404 returned error can't find the container with id 73d132a7233c44110712bdf3766d026892346b93c74827c02db940931e390442 Jul 10 02:31:56.535693 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:56.535661 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" event={"ID":"fe765ea0-799d-47c2-bd14-4b0327dfb386","Type":"ContainerStarted","Data":"73d132a7233c44110712bdf3766d026892346b93c74827c02db940931e390442"} Jul 10 02:31:57.546670 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:57.546636 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" event={"ID":"fe765ea0-799d-47c2-bd14-4b0327dfb386","Type":"ContainerStarted","Data":"0335c9b629fb2aadcc0e8b6d9fcc17ff2836d14898144d1bb79516f9539655b8"} Jul 10 02:31:57.574305 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:31:57.574218 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" podStartSLOduration=1.559420486 podStartE2EDuration="2.574200957s" totalImagesPullingTime="1.014780471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:31:57.572950016 +0000 UTC m=+544.849739463" watchObservedRunningTime="2026-07-10 02:31:57.574200957 +0000 UTC m=+544.850990405" Jul 10 02:32:05.650318 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:05.650285 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:32:05.650873 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:05.650347 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:32:05.662571 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:05.662545 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:32:06.595382 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:06.595355 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z6qxz" Jul 10 02:32:53.114923 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:53.114884 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:32:53.116759 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:32:53.116741 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:37:53.140027 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:37:53.139951 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:37:53.142915 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:37:53.142893 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:42:53.166425 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:42:53.166395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:42:53.170012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:42:53.169991 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:45:00.147225 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.147186 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:45:00.150806 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.150781 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" Jul 10 02:45:00.153620 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.153596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chdf\"" Jul 10 02:45:00.160749 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.160722 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" Jul 10 02:45:00.165521 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.165485 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:45:00.294289 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.294250 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:45:00.296862 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:45:00.296835 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd168d172_8a16_4262_84b0_5b8c6b983f30.slice/crio-88c4b3e558b4235b26a6c1f0be87e8ce931e0cee101d5943aa2571249657d248 WatchSource:0}: Error finding container 88c4b3e558b4235b26a6c1f0be87e8ce931e0cee101d5943aa2571249657d248: Status 404 returned error can't find the container with id 88c4b3e558b4235b26a6c1f0be87e8ce931e0cee101d5943aa2571249657d248 Jul 10 02:45:00.299163 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.299148 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:45:00.935858 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:00.935816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" event={"ID":"d168d172-8a16-4262-84b0-5b8c6b983f30","Type":"ContainerStarted","Data":"88c4b3e558b4235b26a6c1f0be87e8ce931e0cee101d5943aa2571249657d248"} Jul 10 02:45:03.951803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:03.951763 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" event={"ID":"d168d172-8a16-4262-84b0-5b8c6b983f30","Type":"ContainerStarted","Data":"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80"} Jul 10 02:45:03.968491 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:45:03.968432 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" podStartSLOduration=1.5539606190000002 podStartE2EDuration="3.968415659s" totalImagesPullingTime="2.41445504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:03.966590281 +0000 UTC m=+1331.243379739" watchObservedRunningTime="2026-07-10 02:45:03.968415659 +0000 UTC m=+1331.245205186" Jul 10 02:46:00.011397 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:00.011362 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:46:00.011996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:00.011591 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" podUID="d168d172-8a16-4262-84b0-5b8c6b983f30" containerName="cleanup" containerID="cri-o://607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80" gracePeriod=30 Jul 10 02:46:30.192918 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.192889 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" Jul 10 02:46:30.342180 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.342086 2560 generic.go:356] "Generic (PLEG): container finished" podID="d168d172-8a16-4262-84b0-5b8c6b983f30" containerID="607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80" exitCode=137 Jul 10 02:46:30.342180 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.342134 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" event={"ID":"d168d172-8a16-4262-84b0-5b8c6b983f30","Type":"ContainerDied","Data":"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80"} Jul 10 02:46:30.342180 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.342140 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" Jul 10 02:46:30.342180 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.342177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75" event={"ID":"d168d172-8a16-4262-84b0-5b8c6b983f30","Type":"ContainerDied","Data":"88c4b3e558b4235b26a6c1f0be87e8ce931e0cee101d5943aa2571249657d248"} Jul 10 02:46:30.342528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.342198 2560 scope.go:117] "RemoveContainer" containerID="607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80" Jul 10 02:46:30.351198 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.351180 2560 scope.go:117] "RemoveContainer" containerID="607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80" Jul 10 02:46:30.351491 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:46:30.351471 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80\": container with ID starting with 607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80 not found: ID does not exist" containerID="607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80" Jul 10 02:46:30.351557 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.351501 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80"} err="failed to get container status \"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80\": rpc error: code = NotFound desc = could not find container \"607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80\": container with ID starting with 607b6dc8e5553e8652b30e8227f20a1ac060cf9bd3aaa7479ca51b1b8e6caf80 not found: ID does not exist" Jul 10 02:46:30.363570 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.363544 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:46:30.368027 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:30.368005 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-jxz75"] Jul 10 02:46:31.224004 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:31.223973 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d168d172-8a16-4262-84b0-5b8c6b983f30" path="/var/lib/kubelet/pods/d168d172-8a16-4262-84b0-5b8c6b983f30/volumes" Jul 10 02:46:55.143463 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.143425 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:46:55.143941 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.143860 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d168d172-8a16-4262-84b0-5b8c6b983f30" containerName="cleanup" Jul 10 02:46:55.143941 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.143876 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d168d172-8a16-4262-84b0-5b8c6b983f30" containerName="cleanup" Jul 10 02:46:55.144063 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.143970 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d168d172-8a16-4262-84b0-5b8c6b983f30" containerName="cleanup" Jul 10 02:46:55.148596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.148575 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.152418 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.152397 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-91f589dd-openshift-default-dockercfg-ltmqg\"" Jul 10 02:46:55.161323 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.161301 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:46:55.212119 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212173 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212219 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212313 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tkgt\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212346 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.212478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.212415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313630 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tkgt\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.313826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313921 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.313955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.314071 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.314319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.314341 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314468 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.314405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.314666 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.314649 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.316039 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.316016 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.316334 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.316315 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.322020 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.321996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.322232 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.322211 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tkgt\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt\") pod \"e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.459625 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.459527 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:55.598938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.598909 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:46:55.600768 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:46:55.600736 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142f7303_0ec8_4609_9e36_dd9b4211635e.slice/crio-3b60c53e53b84bb5f77b8ca6ed0dbd175a8cc2d0546d2ae27811cbf8cafa072d WatchSource:0}: Error finding container 3b60c53e53b84bb5f77b8ca6ed0dbd175a8cc2d0546d2ae27811cbf8cafa072d: Status 404 returned error can't find the container with id 3b60c53e53b84bb5f77b8ca6ed0dbd175a8cc2d0546d2ae27811cbf8cafa072d Jul 10 02:46:55.602981 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.602944 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:46:55.603051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.603008 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:46:55.603051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:55.603035 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:46:56.461988 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:56.461946 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" event={"ID":"142f7303-0ec8-4609-9e36-dd9b4211635e","Type":"ContainerStarted","Data":"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979"} Jul 10 02:46:56.462404 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:56.462010 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" event={"ID":"142f7303-0ec8-4609-9e36-dd9b4211635e","Type":"ContainerStarted","Data":"3b60c53e53b84bb5f77b8ca6ed0dbd175a8cc2d0546d2ae27811cbf8cafa072d"} Jul 10 02:46:56.487479 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:56.487429 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" podStartSLOduration=1.48741184 podStartE2EDuration="1.48741184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:56.485844467 +0000 UTC m=+1443.762633936" watchObservedRunningTime="2026-07-10 02:46:56.48741184 +0000 UTC m=+1443.764201286" Jul 10 02:46:57.460531 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:57.460486 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:57.465683 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:57.465655 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:57.466660 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:57.466636 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:46:57.467446 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:46:57.467430 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:47:01.510479 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:01.510441 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:47:01.510986 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:01.510659 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" podUID="142f7303-0ec8-4609-9e36-dd9b4211635e" containerName="istio-proxy" containerID="cri-o://eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979" gracePeriod=30 Jul 10 02:47:06.769163 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.769098 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:47:06.822651 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.822842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822672 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.822842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822697 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tkgt\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.822842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822721 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.822842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822744 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.822842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822783 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.823090 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822945 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:06.823090 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.822983 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.823090 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823023 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data" (OuterVolumeSpecName: "istio-data") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:06.823253 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823101 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.823253 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823129 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy\") pod \"142f7303-0ec8-4609-9e36-dd9b4211635e\" (UID: \"142f7303-0ec8-4609-9e36-dd9b4211635e\") " Jul 10 02:47:06.823253 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823131 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:06.823253 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823225 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:06.823464 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823303 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:47:06.823524 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823483 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.823524 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823502 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/142f7303-0ec8-4609-9e36-dd9b4211635e-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.823524 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823515 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.823671 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823526 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.823671 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.823539 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.825198 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.825171 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:06.825339 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.825291 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt" (OuterVolumeSpecName: "kube-api-access-4tkgt") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "kube-api-access-4tkgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:06.825339 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.825307 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:47:06.825425 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.825348 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token" (OuterVolumeSpecName: "istio-token") pod "142f7303-0ec8-4609-9e36-dd9b4211635e" (UID: "142f7303-0ec8-4609-9e36-dd9b4211635e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:06.924966 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.924924 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tkgt\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-kube-api-access-4tkgt\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.924966 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.924959 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.924966 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.924968 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:06.924966 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:06.924978 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/142f7303-0ec8-4609-9e36-dd9b4211635e-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:07.512208 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.512175 2560 generic.go:356] "Generic (PLEG): container finished" podID="142f7303-0ec8-4609-9e36-dd9b4211635e" containerID="eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979" exitCode=0 Jul 10 02:47:07.512405 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.512216 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" event={"ID":"142f7303-0ec8-4609-9e36-dd9b4211635e","Type":"ContainerDied","Data":"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979"} Jul 10 02:47:07.512405 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.512259 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" event={"ID":"142f7303-0ec8-4609-9e36-dd9b4211635e","Type":"ContainerDied","Data":"3b60c53e53b84bb5f77b8ca6ed0dbd175a8cc2d0546d2ae27811cbf8cafa072d"} Jul 10 02:47:07.512405 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.512257 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj" Jul 10 02:47:07.512405 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.512334 2560 scope.go:117] "RemoveContainer" containerID="eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979" Jul 10 02:47:07.521243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.521217 2560 scope.go:117] "RemoveContainer" containerID="eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979" Jul 10 02:47:07.521571 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:47:07.521548 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979\": container with ID starting with eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979 not found: ID does not exist" containerID="eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979" Jul 10 02:47:07.521653 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.521580 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979"} err="failed to get container status \"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979\": rpc error: code = NotFound desc = could not find container \"eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979\": container with ID starting with eccf0b99d15364567017eb9474f1672d1b5f56eb2bd4d076beb08cdf2cad1979 not found: ID does not exist" Jul 10 02:47:07.537429 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.537402 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:47:07.543446 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.543422 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-91f589dd-openshift-default-66b9888c87-47zfj"] Jul 10 02:47:07.727812 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.727773 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:07.728346 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.728323 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142f7303-0ec8-4609-9e36-dd9b4211635e" containerName="istio-proxy" Jul 10 02:47:07.728346 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.728347 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="142f7303-0ec8-4609-9e36-dd9b4211635e" containerName="istio-proxy" Jul 10 02:47:07.728519 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.728480 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="142f7303-0ec8-4609-9e36-dd9b4211635e" containerName="istio-proxy" Jul 10 02:47:07.733224 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.733199 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.736085 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.736064 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8f3ec1ad-openshift-default-dockercfg-bk8tr\"" Jul 10 02:47:07.746428 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.746403 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:07.831331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831356 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9n57\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831508 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.831730 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.831571 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932638 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932660 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n57\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.932864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.933195 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.932982 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.933195 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.933042 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.933353 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.933307 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.933428 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.933335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.933487 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.933424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.935196 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.935178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.935434 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.935418 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.943738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.943709 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:07.943843 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:07.943747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9n57\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57\") pod \"e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:08.046626 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.046587 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:08.193426 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.193373 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:08.197179 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:47:08.197149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod608fe715_b2d0_4dc1_a38c_69f0f09141df.slice/crio-8ea731a0004710f1515fac509a184fd88d7d9633e801ffeec2b3f0bfe16c780b WatchSource:0}: Error finding container 8ea731a0004710f1515fac509a184fd88d7d9633e801ffeec2b3f0bfe16c780b: Status 404 returned error can't find the container with id 8ea731a0004710f1515fac509a184fd88d7d9633e801ffeec2b3f0bfe16c780b Jul 10 02:47:08.199381 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.199344 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:08.199458 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.199421 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:08.199458 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.199454 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:08.519116 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.519072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" event={"ID":"608fe715-b2d0-4dc1-a38c-69f0f09141df","Type":"ContainerStarted","Data":"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788"} Jul 10 02:47:08.519345 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.519147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" event={"ID":"608fe715-b2d0-4dc1-a38c-69f0f09141df","Type":"ContainerStarted","Data":"8ea731a0004710f1515fac509a184fd88d7d9633e801ffeec2b3f0bfe16c780b"} Jul 10 02:47:08.538813 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:08.538754 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" podStartSLOduration=1.538735769 podStartE2EDuration="1.538735769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:08.537101348 +0000 UTC m=+1455.813890813" watchObservedRunningTime="2026-07-10 02:47:08.538735769 +0000 UTC m=+1455.815525215" Jul 10 02:47:09.047750 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:09.047714 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:09.049351 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:09.049322 2560 patch_prober.go:28] interesting pod/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 10 02:47:09.049476 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:09.049377 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 10 02:47:09.225318 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:09.225283 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="142f7303-0ec8-4609-9e36-dd9b4211635e" path="/var/lib/kubelet/pods/142f7303-0ec8-4609-9e36-dd9b4211635e/volumes" Jul 10 02:47:10.051739 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:10.051705 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:10.535291 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:10.534308 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:10.537751 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:10.537718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:19.992318 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:19.992246 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:19.992801 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:19.992510 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerName="istio-proxy" containerID="cri-o://470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788" gracePeriod=30 Jul 10 02:47:22.372440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.372412 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:22.467791 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467750 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.467791 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467794 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467817 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467841 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467882 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467907 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468049 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.467944 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468113 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468217 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9n57\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57\") pod \"608fe715-b2d0-4dc1-a38c-69f0f09141df\" (UID: \"608fe715-b2d0-4dc1-a38c-69f0f09141df\") " Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468321 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468571 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/608fe715-b2d0-4dc1-a38c-69f0f09141df-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468604 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468697 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data" (OuterVolumeSpecName: "istio-data") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:22.468732 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468711 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:22.469050 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.468949 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:22.470367 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.470340 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:47:22.470552 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.470529 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57" (OuterVolumeSpecName: "kube-api-access-f9n57") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "kube-api-access-f9n57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:22.470639 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.470617 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token" (OuterVolumeSpecName: "istio-token") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:22.470687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.470625 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "608fe715-b2d0-4dc1-a38c-69f0f09141df" (UID: "608fe715-b2d0-4dc1-a38c-69f0f09141df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:22.569581 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569541 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9n57\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-kube-api-access-f9n57\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569581 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569575 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569581 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569585 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569594 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569603 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569612 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569619 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.569822 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.569626 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/608fe715-b2d0-4dc1-a38c-69f0f09141df-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:22.600135 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.600093 2560 generic.go:356] "Generic (PLEG): container finished" podID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerID="470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788" exitCode=0 Jul 10 02:47:22.600331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.600159 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" Jul 10 02:47:22.600331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.600180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" event={"ID":"608fe715-b2d0-4dc1-a38c-69f0f09141df","Type":"ContainerDied","Data":"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788"} Jul 10 02:47:22.600331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.600260 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz" event={"ID":"608fe715-b2d0-4dc1-a38c-69f0f09141df","Type":"ContainerDied","Data":"8ea731a0004710f1515fac509a184fd88d7d9633e801ffeec2b3f0bfe16c780b"} Jul 10 02:47:22.600331 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.600304 2560 scope.go:117] "RemoveContainer" containerID="470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788" Jul 10 02:47:22.609529 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.609510 2560 scope.go:117] "RemoveContainer" containerID="470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788" Jul 10 02:47:22.609819 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:47:22.609797 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788\": container with ID starting with 470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788 not found: ID does not exist" containerID="470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788" Jul 10 02:47:22.609866 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.609828 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788"} err="failed to get container status \"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788\": rpc error: code = NotFound desc = could not find container \"470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788\": container with ID starting with 470520424ed91e70979ed8824f4198a028ff015729c0894a4eea26dc7323f788 not found: ID does not exist" Jul 10 02:47:22.623967 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.623936 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:22.631315 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:22.631285 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8f3ec1ad-openshift-default-9d6696d97-qdbwz"] Jul 10 02:47:23.225165 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:23.225129 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" path="/var/lib/kubelet/pods/608fe715-b2d0-4dc1-a38c-69f0f09141df/volumes" Jul 10 02:47:26.085569 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.085529 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:26.086091 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.086068 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerName="istio-proxy" Jul 10 02:47:26.086176 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.086094 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerName="istio-proxy" Jul 10 02:47:26.086235 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.086175 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="608fe715-b2d0-4dc1-a38c-69f0f09141df" containerName="istio-proxy" Jul 10 02:47:26.088738 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.088715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.091118 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.091096 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-851fc22b-openshift-default-dockercfg-ndsd5\"" Jul 10 02:47:26.104261 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.104229 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:26.201297 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201501 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201575 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwdd2\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201644 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201742 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201969 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201969 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201884 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201969 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.201969 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.201965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303402 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303627 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303627 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303749 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303749 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303693 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303749 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwdd2\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303781 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.303890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.304119 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.303934 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.304366 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.304330 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.304534 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.304514 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.306219 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.306196 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.306547 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.306529 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.312440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.312411 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.312826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.312804 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwdd2\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2\") pod \"e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.400623 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.400527 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:26.549327 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.549295 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:26.550622 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:47:26.550594 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab9d590b_d744_42f6_88bb_34c7338a3610.slice/crio-617bc696d9f5d224b33088ebaf55c16fb889c3bcd376a883b59852051743aa44 WatchSource:0}: Error finding container 617bc696d9f5d224b33088ebaf55c16fb889c3bcd376a883b59852051743aa44: Status 404 returned error can't find the container with id 617bc696d9f5d224b33088ebaf55c16fb889c3bcd376a883b59852051743aa44 Jul 10 02:47:26.552752 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.552723 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:26.552839 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.552790 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:26.552839 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.552823 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:47:26.621680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.621646 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" event={"ID":"ab9d590b-d744-42f6-88bb-34c7338a3610","Type":"ContainerStarted","Data":"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695"} Jul 10 02:47:26.621832 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.621707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" event={"ID":"ab9d590b-d744-42f6-88bb-34c7338a3610","Type":"ContainerStarted","Data":"617bc696d9f5d224b33088ebaf55c16fb889c3bcd376a883b59852051743aa44"} Jul 10 02:47:26.642084 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:26.642031 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" podStartSLOduration=0.642010106 podStartE2EDuration="642.010106ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:26.640443742 +0000 UTC m=+1473.917233188" watchObservedRunningTime="2026-07-10 02:47:26.642010106 +0000 UTC m=+1473.918799552" Jul 10 02:47:27.401697 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:27.401661 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:27.406687 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:27.406655 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:27.626576 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:27.626543 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:27.627441 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:27.627422 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:31.800649 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:31.800611 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:31.801086 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:31.800840 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" podUID="ab9d590b-d744-42f6-88bb-34c7338a3610" containerName="istio-proxy" containerID="cri-o://24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695" gracePeriod=30 Jul 10 02:47:37.045406 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.045377 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:37.100741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100641 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.100741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100685 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.100741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100700 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.100741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100722 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100751 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwdd2\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100830 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100861 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo\") pod \"ab9d590b-d744-42f6-88bb-34c7338a3610\" (UID: \"ab9d590b-d744-42f6-88bb-34c7338a3610\") " Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.100997 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:37.101067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101015 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101141 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101178 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101322 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101371 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101392 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101406 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab9d590b-d744-42f6-88bb-34c7338a3610-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.101417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.101419 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.103181 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.103152 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:47:37.103323 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.103194 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:37.103323 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.103212 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:37.103323 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.103228 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2" (OuterVolumeSpecName: "kube-api-access-wwdd2") pod "ab9d590b-d744-42f6-88bb-34c7338a3610" (UID: "ab9d590b-d744-42f6-88bb-34c7338a3610"). InnerVolumeSpecName "kube-api-access-wwdd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:37.201792 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.201759 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.201792 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.201790 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.201987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.201805 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.201987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.201816 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab9d590b-d744-42f6-88bb-34c7338a3610-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.201987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.201828 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwdd2\" (UniqueName: \"kubernetes.io/projected/ab9d590b-d744-42f6-88bb-34c7338a3610-kube-api-access-wwdd2\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:47:37.673797 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.673766 2560 generic.go:356] "Generic (PLEG): container finished" podID="ab9d590b-d744-42f6-88bb-34c7338a3610" containerID="24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695" exitCode=0 Jul 10 02:47:37.673987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.673807 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" Jul 10 02:47:37.673987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.673860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" event={"ID":"ab9d590b-d744-42f6-88bb-34c7338a3610","Type":"ContainerDied","Data":"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695"} Jul 10 02:47:37.673987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.673918 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6" event={"ID":"ab9d590b-d744-42f6-88bb-34c7338a3610","Type":"ContainerDied","Data":"617bc696d9f5d224b33088ebaf55c16fb889c3bcd376a883b59852051743aa44"} Jul 10 02:47:37.673987 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.673941 2560 scope.go:117] "RemoveContainer" containerID="24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695" Jul 10 02:47:37.683911 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.683874 2560 scope.go:117] "RemoveContainer" containerID="24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695" Jul 10 02:47:37.684280 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:47:37.684236 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695\": container with ID starting with 24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695 not found: ID does not exist" containerID="24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695" Jul 10 02:47:37.684386 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.684295 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695"} err="failed to get container status \"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695\": rpc error: code = NotFound desc = could not find container \"24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695\": container with ID starting with 24a9f29e3d4161ea62b1a858a6630407ce6dc541b9adc0cc8b7a8dd2cdeda695 not found: ID does not exist" Jul 10 02:47:37.692410 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.692376 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:37.701314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:37.701259 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-851fc22b-openshift-default-c79c5b99-cswd6"] Jul 10 02:47:39.224568 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:39.224532 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9d590b-d744-42f6-88bb-34c7338a3610" path="/var/lib/kubelet/pods/ab9d590b-d744-42f6-88bb-34c7338a3610/volumes" Jul 10 02:47:53.192376 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:53.192346 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:47:53.197750 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:47:53.197725 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:49:15.421601 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.421561 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:15.422061 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.421902 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9d590b-d744-42f6-88bb-34c7338a3610" containerName="istio-proxy" Jul 10 02:49:15.422061 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.421914 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9d590b-d744-42f6-88bb-34c7338a3610" containerName="istio-proxy" Jul 10 02:49:15.422061 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.421984 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9d590b-d744-42f6-88bb-34c7338a3610" containerName="istio-proxy" Jul 10 02:49:15.424862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.424846 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.427984 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.427962 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a1436a69-openshift-default-dockercfg-2hd44\"" Jul 10 02:49:15.454141 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.454119 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:15.520508 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520508 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520614 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520657 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520741 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520678 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.520908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.520863 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kgzj\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622228 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kgzj\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622387 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622240 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622387 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622258 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622387 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622387 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622631 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622669 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622722 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622682 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.622722 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.622686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.623034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.623013 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.623136 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.623030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.623295 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.623253 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.623477 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.623456 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.624664 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.624643 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.624664 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.624656 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.631592 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.631549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.632742 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.632719 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kgzj\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj\") pod \"e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.734676 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.734603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:15.878673 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.878644 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:15.881816 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:49:15.881790 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f8c666_3c01_4459_b265_6fc465e92b9b.slice/crio-405e7c06ff9ac90db863cc3c71416f54601241db20c998ae2781fb19d37d2a81 WatchSource:0}: Error finding container 405e7c06ff9ac90db863cc3c71416f54601241db20c998ae2781fb19d37d2a81: Status 404 returned error can't find the container with id 405e7c06ff9ac90db863cc3c71416f54601241db20c998ae2781fb19d37d2a81 Jul 10 02:49:15.883859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.883827 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:15.883939 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.883892 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:15.883939 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:15.883917 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:16.119862 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:16.119825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerStarted","Data":"a1e583606a1d37d65260acb51d07cbbe4375e2d5537832c09da73d0e1b485a36"} Jul 10 02:49:16.120025 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:16.119882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerStarted","Data":"405e7c06ff9ac90db863cc3c71416f54601241db20c998ae2781fb19d37d2a81"} Jul 10 02:49:16.150910 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:16.150846 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podStartSLOduration=1.1508317 podStartE2EDuration="1.1508317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:16.148681431 +0000 UTC m=+1583.425470879" watchObservedRunningTime="2026-07-10 02:49:16.1508317 +0000 UTC m=+1583.427621146" Jul 10 02:49:16.735448 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:16.735411 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:16.739968 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:16.739946 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:17.125608 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:17.125583 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:17.126591 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:17.126572 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:31.197024 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.196988 2560 generic.go:356] "Generic (PLEG): container finished" podID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerID="a1e583606a1d37d65260acb51d07cbbe4375e2d5537832c09da73d0e1b485a36" exitCode=0 Jul 10 02:49:31.197478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.197060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerDied","Data":"a1e583606a1d37d65260acb51d07cbbe4375e2d5537832c09da73d0e1b485a36"} Jul 10 02:49:31.197478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.197446 2560 scope.go:117] "RemoveContainer" containerID="a1e583606a1d37d65260acb51d07cbbe4375e2d5537832c09da73d0e1b485a36" Jul 10 02:49:31.198491 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.198464 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:31.198626 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.198609 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:31.198680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:31.198654 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:32.126051 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:32.126010 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:32.203242 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:32.203206 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerStarted","Data":"b079aaf46e17f192da97e407a3b5846d2bef859afe919af522754f919e69f5dc"} Jul 10 02:49:32.735126 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:32.735088 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:32.736122 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:32.736096 2560 patch_prober.go:28] interesting pod/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 10 02:49:32.736252 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:32.736156 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 10 02:49:33.735667 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:33.735628 2560 patch_prober.go:28] interesting pod/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 10 02:49:33.736142 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:33.735699 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 10 02:49:34.735953 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:34.735920 2560 patch_prober.go:28] interesting pod/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 10 02:49:34.736383 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:34.735974 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 10 02:49:35.735751 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:35.735717 2560 patch_prober.go:28] interesting pod/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 10 02:49:35.735926 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:35.735767 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 10 02:49:36.494588 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:36.494553 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:36.495034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:36.494734 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" containerID="cri-o://b079aaf46e17f192da97e407a3b5846d2bef859afe919af522754f919e69f5dc" gracePeriod=30 Jul 10 02:49:42.252048 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.252012 2560 generic.go:356] "Generic (PLEG): container finished" podID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerID="b079aaf46e17f192da97e407a3b5846d2bef859afe919af522754f919e69f5dc" exitCode=0 Jul 10 02:49:42.252452 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.252053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerDied","Data":"b079aaf46e17f192da97e407a3b5846d2bef859afe919af522754f919e69f5dc"} Jul 10 02:49:42.252452 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.252102 2560 scope.go:117] "RemoveContainer" containerID="a1e583606a1d37d65260acb51d07cbbe4375e2d5537832c09da73d0e1b485a36" Jul 10 02:49:42.338859 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.338836 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:42.450817 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450753 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.450817 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450786 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.450817 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450806 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450822 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kgzj\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450863 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450890 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450949 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451059 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.450973 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert\") pod \"69f8c666-3c01-4459-b265-6fc465e92b9b\" (UID: \"69f8c666-3c01-4459-b265-6fc465e92b9b\") " Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451040 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451100 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451115 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451261 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451308 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451321 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.451377 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451242 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:49:42.451650 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.451414 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:49:42.453039 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.453018 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:49:42.453156 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.453127 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:49:42.453493 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.453475 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:49:42.453565 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.453502 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj" (OuterVolumeSpecName: "kube-api-access-2kgzj") pod "69f8c666-3c01-4459-b265-6fc465e92b9b" (UID: "69f8c666-3c01-4459-b265-6fc465e92b9b"). InnerVolumeSpecName "kube-api-access-2kgzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:49:42.552670 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552644 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.552670 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552666 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.552670 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552676 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.552861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552685 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.552861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552692 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f8c666-3c01-4459-b265-6fc465e92b9b-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:42.552861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:42.552701 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kgzj\" (UniqueName: \"kubernetes.io/projected/69f8c666-3c01-4459-b265-6fc465e92b9b-kube-api-access-2kgzj\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:49:43.257328 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:43.257294 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" event={"ID":"69f8c666-3c01-4459-b265-6fc465e92b9b","Type":"ContainerDied","Data":"405e7c06ff9ac90db863cc3c71416f54601241db20c998ae2781fb19d37d2a81"} Jul 10 02:49:43.257662 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:43.257351 2560 scope.go:117] "RemoveContainer" containerID="b079aaf46e17f192da97e407a3b5846d2bef859afe919af522754f919e69f5dc" Jul 10 02:49:43.257662 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:43.257306 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g" Jul 10 02:49:43.275768 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:43.275743 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:43.279185 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:43.279165 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a1436a69-openshift-default-669f67f7f5-bdx6g"] Jul 10 02:49:45.224133 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:45.224062 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" path="/var/lib/kubelet/pods/69f8c666-3c01-4459-b265-6fc465e92b9b/volumes" Jul 10 02:49:56.083341 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.083310 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:49:56.083762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.083656 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:49:56.083762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.083668 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:49:56.083762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.083680 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:49:56.083762 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.083685 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:49:56.087286 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.084082 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:49:56.088087 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.088065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.092148 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.092130 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dc829df0-openshift-default-dockercfg-cvqhj\"" Jul 10 02:49:56.101978 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.101957 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:49:56.156956 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.156912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157098 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.156971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj8gn\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157098 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157098 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157098 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157083 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157166 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.157247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.157222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258316 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258326 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258437 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258500 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258754 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258754 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258559 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj8gn\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258754 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258754 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258697 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258975 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258806 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.258975 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.258881 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.259074 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.259040 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.259283 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.259247 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.261080 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.261058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.261167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.261087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.267841 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.267816 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj8gn\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.267937 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.267820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token\") pod \"e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.398391 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.398337 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:56.534392 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.534366 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:49:56.536056 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:49:56.536035 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c63eeeb_c9b6_475e_831e_1e9f2815f39c.slice/crio-a44e5aceaf2e5069cacc363366d8d6675cdfe8cd340e3e2223ba1aa10b4860eb WatchSource:0}: Error finding container a44e5aceaf2e5069cacc363366d8d6675cdfe8cd340e3e2223ba1aa10b4860eb: Status 404 returned error can't find the container with id a44e5aceaf2e5069cacc363366d8d6675cdfe8cd340e3e2223ba1aa10b4860eb Jul 10 02:49:56.538643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.538598 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:56.538711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.538697 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:56.538761 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:56.538745 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:49:57.318874 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:57.318838 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerStarted","Data":"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848"} Jul 10 02:49:57.319225 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:57.318889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerStarted","Data":"a44e5aceaf2e5069cacc363366d8d6675cdfe8cd340e3e2223ba1aa10b4860eb"} Jul 10 02:49:57.344616 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:57.344563 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podStartSLOduration=1.344548193 podStartE2EDuration="1.344548193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:57.34235214 +0000 UTC m=+1624.619141586" watchObservedRunningTime="2026-07-10 02:49:57.344548193 +0000 UTC m=+1624.621337638" Jul 10 02:49:57.398638 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:57.398614 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:57.403243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:57.403222 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:58.322770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:58.322740 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:49:58.323667 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:49:58.323647 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:50:12.393709 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.393673 2560 generic.go:356] "Generic (PLEG): container finished" podID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerID="7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848" exitCode=0 Jul 10 02:50:12.394171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.393754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerDied","Data":"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848"} Jul 10 02:50:12.394171 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.394078 2560 scope.go:117] "RemoveContainer" containerID="7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848" Jul 10 02:50:12.394957 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.394934 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:50:12.395018 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.395007 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:50:12.395058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:12.395037 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:50:13.323679 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:13.323644 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:50:13.399865 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:13.399832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerStarted","Data":"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b"} Jul 10 02:50:14.398967 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:14.398926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:50:14.400023 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:14.399999 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 10 02:50:14.400367 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:14.400081 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 10 02:50:15.399145 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:15.399110 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 10 02:50:15.399335 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:15.399171 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 10 02:50:16.399280 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:16.399233 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 10 02:50:16.399616 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:16.399302 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 10 02:50:17.399209 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:17.399176 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 10 02:50:17.399402 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:17.399243 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 10 02:50:18.399136 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:18.399104 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 10 02:50:18.399329 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:18.399167 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 10 02:50:19.402318 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:19.402290 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:50:19.402706 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:19.402543 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:50:19.403091 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:50:19.403075 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:51:46.122131 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:46.122096 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:51:46.122716 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:46.122330 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" containerID="cri-o://ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b" gracePeriod=30 Jul 10 02:51:49.405243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:49.405204 2560 patch_prober.go:28] interesting pod/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 02:51:49.405610 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:49.405255 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 02:51:51.380990 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.380968 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:51:51.489413 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489345 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489413 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489388 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489413 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489404 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489437 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj8gn\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489498 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489531 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489556 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489595 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy\") pod \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\" (UID: \"8c63eeeb-c9b6-475e-831e-1e9f2815f39c\") " Jul 10 02:51:51.489889 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489628 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:51:51.489889 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.489697 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:51:51.490644 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.490513 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:51:51.490644 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.490543 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.490644 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.490566 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.490644 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.490639 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:51:51.493314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.490950 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:51:51.493314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.492479 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:51:51.493314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.492499 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:51:51.493314 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.492529 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn" (OuterVolumeSpecName: "kube-api-access-qj8gn") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "kube-api-access-qj8gn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:51:51.497783 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.497761 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c63eeeb-c9b6-475e-831e-1e9f2815f39c" (UID: "8c63eeeb-c9b6-475e-831e-1e9f2815f39c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:51:51.591852 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591817 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.591852 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591848 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj8gn\" (UniqueName: \"kubernetes.io/projected/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-kube-api-access-qj8gn\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.591852 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591858 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.592107 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591866 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.592107 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591874 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.592107 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591883 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.592107 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.591891 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c63eeeb-c9b6-475e-831e-1e9f2815f39c-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:51:51.826584 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.826550 2560 generic.go:356] "Generic (PLEG): container finished" podID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerID="ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b" exitCode=0 Jul 10 02:51:51.826771 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.826601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerDied","Data":"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b"} Jul 10 02:51:51.826771 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.826620 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" Jul 10 02:51:51.826771 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.826667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq" event={"ID":"8c63eeeb-c9b6-475e-831e-1e9f2815f39c","Type":"ContainerDied","Data":"a44e5aceaf2e5069cacc363366d8d6675cdfe8cd340e3e2223ba1aa10b4860eb"} Jul 10 02:51:51.826771 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.826688 2560 scope.go:117] "RemoveContainer" containerID="ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b" Jul 10 02:51:51.835144 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.835123 2560 scope.go:117] "RemoveContainer" containerID="7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848" Jul 10 02:51:51.842818 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.842802 2560 scope.go:117] "RemoveContainer" containerID="ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b" Jul 10 02:51:51.843033 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:51:51.843018 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b\": container with ID starting with ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b not found: ID does not exist" containerID="ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b" Jul 10 02:51:51.843083 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.843040 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b"} err="failed to get container status \"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b\": rpc error: code = NotFound desc = could not find container \"ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b\": container with ID starting with ed803934e48937b5fef2e04c6ef4c2e27c1ef8cc1f85bc6deeae2dffb95e765b not found: ID does not exist" Jul 10 02:51:51.843083 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.843056 2560 scope.go:117] "RemoveContainer" containerID="7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848" Jul 10 02:51:51.843245 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:51:51.843225 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848\": container with ID starting with 7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848 not found: ID does not exist" containerID="7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848" Jul 10 02:51:51.843325 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.843255 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848"} err="failed to get container status \"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848\": rpc error: code = NotFound desc = could not find container \"7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848\": container with ID starting with 7ff20cc7b5d7e0e9bb9e83b836d7c5c5f75345ef140da6d1a765f19e46aee848 not found: ID does not exist" Jul 10 02:51:51.849085 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.849065 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:51:51.852975 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:51.852955 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dc829df0-openshift-default-76b9dd6d7-v2blq"] Jul 10 02:51:53.225635 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:51:53.225605 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" path="/var/lib/kubelet/pods/8c63eeeb-c9b6-475e-831e-1e9f2815f39c/volumes" Jul 10 02:52:41.571375 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571341 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:52:41.571803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571660 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:41.571803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571672 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:41.571803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571755 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f8c666-3c01-4459-b265-6fc465e92b9b" containerName="istio-proxy" Jul 10 02:52:41.571803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571766 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:41.571803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.571774 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:41.574590 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.574576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.576955 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.576925 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-829dac-openshift-default-dockercfg-wp7qm\"" Jul 10 02:52:41.587626 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.587602 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:52:41.669258 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669224 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669430 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669430 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669430 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669430 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669360 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669636 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669636 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669636 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rngt8\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.669636 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.669602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770243 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770381 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rngt8\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770783 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770783 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770783 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770731 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770914 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770831 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.770953 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.770916 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.771117 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.771098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.772903 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.772882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.773037 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.773018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.778540 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.778518 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.779328 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.779309 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rngt8\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8\") pod \"e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:41.884603 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:41.884547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:42.023948 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:42.023792 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:52:42.026642 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:52:42.026618 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd097f714_e278_4b2d_a55c_d3173e1db071.slice/crio-e785250b5c4314610cfd1e79f6917f2f7ea177c710b8b4cf4169797551307d3b WatchSource:0}: Error finding container e785250b5c4314610cfd1e79f6917f2f7ea177c710b8b4cf4169797551307d3b: Status 404 returned error can't find the container with id e785250b5c4314610cfd1e79f6917f2f7ea177c710b8b4cf4169797551307d3b Jul 10 02:52:42.028624 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:42.028594 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:42.028722 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:42.028654 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:42.028722 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:42.028686 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:42.053372 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:42.053347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerStarted","Data":"e785250b5c4314610cfd1e79f6917f2f7ea177c710b8b4cf4169797551307d3b"} Jul 10 02:52:43.058857 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:43.058822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerStarted","Data":"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f"} Jul 10 02:52:43.080320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:43.080254 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podStartSLOduration=2.08024051 podStartE2EDuration="2.08024051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:52:43.079037088 +0000 UTC m=+1790.355826555" watchObservedRunningTime="2026-07-10 02:52:43.08024051 +0000 UTC m=+1790.357029955" Jul 10 02:52:43.884936 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:43.884900 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:43.889102 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:43.889076 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:44.063802 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:44.063768 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:44.064611 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:44.064594 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:50.070636 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.070553 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:52:50.071108 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.071092 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:50.071153 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.071111 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c63eeeb-c9b6-475e-831e-1e9f2815f39c" containerName="istio-proxy" Jul 10 02:52:50.074718 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.074695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.077981 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.077961 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1bc4ca-openshift-default-dockercfg-tswh7\"" Jul 10 02:52:50.083913 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.083890 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:52:50.255182 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.255140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g79qq\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.255508 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.255487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.255661 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.255646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.255794 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.255780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.255932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.255918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.256071 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.256056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.256201 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.256187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.256409 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.256391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.256552 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.256537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357172 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357437 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357573 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357704 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357833 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.357955 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.357941 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.358132 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.358117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.358237 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.358223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.358391 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.358375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g79qq\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.359547 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.359526 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.359921 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.359901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.360033 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.359991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.360286 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.360244 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.360920 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.360899 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.363527 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.363507 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.363942 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.363888 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.367320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.367294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g79qq\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.367876 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.367859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token\") pod \"e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.387615 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.387597 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:50.549678 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.549594 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:52:50.565374 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.564103 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:50.565374 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.564181 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:50.565374 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:50.564223 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:51.095707 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.095668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerStarted","Data":"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854"} Jul 10 02:52:51.096088 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.095732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerStarted","Data":"9b8f8b72b70a4fd8143f0febca98d652478da8303d7a65e750d1bd356c629bbc"} Jul 10 02:52:51.116154 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.116110 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podStartSLOduration=1.116092721 podStartE2EDuration="1.116092721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:52:51.113812619 +0000 UTC m=+1798.390602064" watchObservedRunningTime="2026-07-10 02:52:51.116092721 +0000 UTC m=+1798.392882215" Jul 10 02:52:51.388157 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.388080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:51.389510 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.389487 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:52:51.389609 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:51.389536 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:52:52.391944 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:52.391917 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:53.104168 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:53.104133 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:53.104956 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:53.104938 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:52:53.218633 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:53.218604 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:52:53.232038 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:53.232015 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:52:58.129630 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.129597 2560 generic.go:356] "Generic (PLEG): container finished" podID="d097f714-e278-4b2d-a55c-d3173e1db071" containerID="844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f" exitCode=0 Jul 10 02:52:58.130033 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.129671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerDied","Data":"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f"} Jul 10 02:52:58.130089 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.130041 2560 scope.go:117] "RemoveContainer" containerID="844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f" Jul 10 02:52:58.130933 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.130910 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:58.130989 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.130980 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:58.131029 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:58.131013 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:52:59.064287 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:59.064210 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:59.135910 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:59.135873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerStarted","Data":"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5"} Jul 10 02:52:59.888133 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:59.888086 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:52:59.888358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:59.888187 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:52:59.888358 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:52:59.888239 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:53:00.885927 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:00.885878 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:53:00.886445 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:00.885948 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:53:01.885364 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:01.885320 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:53:01.885587 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:01.885389 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:53:02.885378 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:02.885335 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:53:02.885880 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:02.885444 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:53:03.885600 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:03.885558 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:53:03.886227 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:03.886181 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:53:04.890698 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:04.890658 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:53:04.891176 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:04.891077 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:53:04.891952 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:04.891931 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:53:07.184584 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.184553 2560 generic.go:356] "Generic (PLEG): container finished" podID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerID="fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854" exitCode=0 Jul 10 02:53:07.184959 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.184627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerDied","Data":"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854"} Jul 10 02:53:07.185007 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.184982 2560 scope.go:117] "RemoveContainer" containerID="fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854" Jul 10 02:53:07.185947 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.185919 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:53:07.186044 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.186003 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:53:07.186108 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:07.186052 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:53:08.104783 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:08.104749 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:53:08.191285 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:08.191240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerStarted","Data":"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3"} Jul 10 02:53:08.388383 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:08.388262 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:53:08.389530 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:08.389499 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:53:08.389663 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:08.389552 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:53:09.388878 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:09.388844 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:53:09.389242 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:09.388894 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:53:10.387924 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:10.387889 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:53:10.388084 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:10.387941 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:53:11.388211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:11.388178 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:53:11.388613 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:11.388234 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:53:12.388643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:12.388607 2560 patch_prober.go:28] interesting pod/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 10 02:53:12.389030 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:12.388659 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 10 02:53:13.391529 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:13.391501 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:53:13.391889 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:13.391790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:53:13.392503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:53:13.392486 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:54:44.877630 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:44.877597 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:54:44.878189 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:44.877864 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" containerID="cri-o://c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5" gracePeriod=30 Jul 10 02:54:49.892802 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:49.892771 2560 patch_prober.go:28] interesting pod/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 10 02:54:49.893174 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:49.892830 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 10 02:54:50.133938 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.133916 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:54:50.248890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.248823 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.248890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.248860 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249052 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.248893 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249052 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.248929 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249052 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.248989 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249052 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249027 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249050 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rngt8\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249085 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249110 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket\") pod \"d097f714-e278-4b2d-a55c-d3173e1db071\" (UID: \"d097f714-e278-4b2d-a55c-d3173e1db071\") " Jul 10 02:54:50.249250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249124 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data" (OuterVolumeSpecName: "istio-data") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:50.249250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249240 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:50.249550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249387 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:54:50.249550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249477 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.249550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249495 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.249550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249509 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d097f714-e278-4b2d-a55c-d3173e1db071-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.249802 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249779 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:50.249868 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.249791 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:50.251097 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.251077 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token" (OuterVolumeSpecName: "istio-token") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:54:50.251210 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.251139 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:54:50.251210 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.251170 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8" (OuterVolumeSpecName: "kube-api-access-rngt8") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "kube-api-access-rngt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:54:50.251304 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.251214 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d097f714-e278-4b2d-a55c-d3173e1db071" (UID: "d097f714-e278-4b2d-a55c-d3173e1db071"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:50.350566 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350545 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.350566 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350565 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.350703 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350574 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.350703 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350583 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d097f714-e278-4b2d-a55c-d3173e1db071-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.350703 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350590 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d097f714-e278-4b2d-a55c-d3173e1db071-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.350703 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.350599 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rngt8\" (UniqueName: \"kubernetes.io/projected/d097f714-e278-4b2d-a55c-d3173e1db071-kube-api-access-rngt8\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:50.645850 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.645815 2560 generic.go:356] "Generic (PLEG): container finished" podID="d097f714-e278-4b2d-a55c-d3173e1db071" containerID="c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5" exitCode=0 Jul 10 02:54:50.646000 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.645868 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" Jul 10 02:54:50.646000 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.645898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerDied","Data":"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5"} Jul 10 02:54:50.646000 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.645956 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z" event={"ID":"d097f714-e278-4b2d-a55c-d3173e1db071","Type":"ContainerDied","Data":"e785250b5c4314610cfd1e79f6917f2f7ea177c710b8b4cf4169797551307d3b"} Jul 10 02:54:50.646000 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.645977 2560 scope.go:117] "RemoveContainer" containerID="c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5" Jul 10 02:54:50.654758 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.654741 2560 scope.go:117] "RemoveContainer" containerID="844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f" Jul 10 02:54:50.664390 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.664343 2560 scope.go:117] "RemoveContainer" containerID="c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5" Jul 10 02:54:50.664878 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:54:50.664852 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5\": container with ID starting with c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5 not found: ID does not exist" containerID="c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5" Jul 10 02:54:50.664964 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.664892 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5"} err="failed to get container status \"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5\": rpc error: code = NotFound desc = could not find container \"c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5\": container with ID starting with c59a928df51a840c5d29c8efd4ca3986ae2341b16b1eba934c73a62098afbba5 not found: ID does not exist" Jul 10 02:54:50.664964 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.664909 2560 scope.go:117] "RemoveContainer" containerID="844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f" Jul 10 02:54:50.665282 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:54:50.665237 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f\": container with ID starting with 844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f not found: ID does not exist" containerID="844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f" Jul 10 02:54:50.665480 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.665436 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f"} err="failed to get container status \"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f\": rpc error: code = NotFound desc = could not find container \"844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f\": container with ID starting with 844eb3db4ca3cadb7eb622fdd101021948cd5d1533b3511b86dae57ee7ffba6f not found: ID does not exist" Jul 10 02:54:50.667815 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.667791 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:54:50.671038 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:50.671016 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-829dac-openshift-default-5f85f48547-8vx5z"] Jul 10 02:54:51.224102 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:51.224070 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" path="/var/lib/kubelet/pods/d097f714-e278-4b2d-a55c-d3173e1db071/volumes" Jul 10 02:54:51.680162 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:51.680125 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:54:51.680465 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:51.680412 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" containerID="cri-o://b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3" gracePeriod=30 Jul 10 02:54:52.521526 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.521487 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:54:52.522515 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522486 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.522515 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522511 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.522705 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522542 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.522705 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522550 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.522705 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522642 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.522705 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.522655 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d097f714-e278-4b2d-a55c-d3173e1db071" containerName="istio-proxy" Jul 10 02:54:52.528464 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.528445 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.530715 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.530695 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6e451c-openshift-default-dockercfg-np2h4\"" Jul 10 02:54:52.534808 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.534788 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:54:52.567661 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567759 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567759 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxj6r\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567929 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567929 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.567929 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.567900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669316 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxj6r\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669711 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669875 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.669875 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.669833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.670101 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.670074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.670205 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.670098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.670205 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.670135 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.670205 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.670161 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.671804 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.671783 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.672094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.672072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.676925 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.676903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.677300 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.677284 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxj6r\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r\") pod \"e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.840160 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.840076 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:52.972220 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.972192 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:54:52.974023 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:54:52.973988 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c32f556_d5ca_465e_aa59_fdf6d15b6062.slice/crio-ab0823f8943cd66fde906295061b72894c1be086bf3fe170a57ae2b23e00ac9c WatchSource:0}: Error finding container ab0823f8943cd66fde906295061b72894c1be086bf3fe170a57ae2b23e00ac9c: Status 404 returned error can't find the container with id ab0823f8943cd66fde906295061b72894c1be086bf3fe170a57ae2b23e00ac9c Jul 10 02:54:52.976116 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.976078 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:54:52.976211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.976159 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:54:52.976211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:52.976203 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:54:53.662550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:53.662469 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerStarted","Data":"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4"} Jul 10 02:54:53.662550 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:53.662514 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerStarted","Data":"ab0823f8943cd66fde906295061b72894c1be086bf3fe170a57ae2b23e00ac9c"} Jul 10 02:54:53.681976 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:53.681929 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podStartSLOduration=1.681913937 podStartE2EDuration="1.681913937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:54:53.681261789 +0000 UTC m=+1920.958051229" watchObservedRunningTime="2026-07-10 02:54:53.681913937 +0000 UTC m=+1920.958703384" Jul 10 02:54:53.841108 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:53.841078 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:54.845380 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:54.845351 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:55.670792 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:55.670762 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:55.671772 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:55.671753 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:54:56.932640 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:56.932616 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:54:57.007182 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007159 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g79qq\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007326 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007211 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007326 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007236 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007326 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007281 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007326 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007302 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007326 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007322 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007566 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007360 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007566 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007387 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007669 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007565 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:57.007669 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007607 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy\") pod \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\" (UID: \"f435b2fb-d990-4a6f-b29a-6abfe42ceef0\") " Jul 10 02:54:57.007763 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007703 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:57.007763 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007716 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:57.007925 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007904 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:57.008012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007973 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.008012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007994 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.008012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.007992 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:54:57.008012 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.008007 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.008192 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.008020 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.009576 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.009548 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:54:57.009842 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.009822 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq" (OuterVolumeSpecName: "kube-api-access-g79qq") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "kube-api-access-g79qq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:54:57.009932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.009847 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:54:57.009932 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.009859 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f435b2fb-d990-4a6f-b29a-6abfe42ceef0" (UID: "f435b2fb-d990-4a6f-b29a-6abfe42ceef0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:54:57.108778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.108746 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.108778 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.108770 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g79qq\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-kube-api-access-g79qq\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.108922 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.108783 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.108922 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.108796 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.108922 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.108805 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f435b2fb-d990-4a6f-b29a-6abfe42ceef0-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:54:57.680561 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.680522 2560 generic.go:356] "Generic (PLEG): container finished" podID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerID="b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3" exitCode=0 Jul 10 02:54:57.680735 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.680570 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" Jul 10 02:54:57.680735 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.680611 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerDied","Data":"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3"} Jul 10 02:54:57.680735 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.680667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk" event={"ID":"f435b2fb-d990-4a6f-b29a-6abfe42ceef0","Type":"ContainerDied","Data":"9b8f8b72b70a4fd8143f0febca98d652478da8303d7a65e750d1bd356c629bbc"} Jul 10 02:54:57.680735 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.680691 2560 scope.go:117] "RemoveContainer" containerID="b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3" Jul 10 02:54:57.689040 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.689021 2560 scope.go:117] "RemoveContainer" containerID="fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854" Jul 10 02:54:57.696401 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.696384 2560 scope.go:117] "RemoveContainer" containerID="b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3" Jul 10 02:54:57.696643 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:54:57.696621 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3\": container with ID starting with b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3 not found: ID does not exist" containerID="b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3" Jul 10 02:54:57.696696 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.696650 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3"} err="failed to get container status \"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3\": rpc error: code = NotFound desc = could not find container \"b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3\": container with ID starting with b05579674f22b0a82acb9ce3566ccedceb70ae919861307c9c3951eab04975d3 not found: ID does not exist" Jul 10 02:54:57.696696 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.696665 2560 scope.go:117] "RemoveContainer" containerID="fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854" Jul 10 02:54:57.696912 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:54:57.696894 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854\": container with ID starting with fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854 not found: ID does not exist" containerID="fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854" Jul 10 02:54:57.696959 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.696919 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854"} err="failed to get container status \"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854\": rpc error: code = NotFound desc = could not find container \"fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854\": container with ID starting with fe27fe695a1d043e0df81e8da8e8d302cae5775bd02303dc4dc93efc640bd854 not found: ID does not exist" Jul 10 02:54:57.700027 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.700005 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:54:57.703940 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:57.703921 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1bc4ca-openshift-default-7748c7f44c-7lddk"] Jul 10 02:54:59.224011 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:54:59.223979 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" path="/var/lib/kubelet/pods/f435b2fb-d990-4a6f-b29a-6abfe42ceef0/volumes" Jul 10 02:55:00.775311 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775254 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:00.775680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775627 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.775680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775638 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.775680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775650 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.775680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775655 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.775814 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775703 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.775814 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.775712 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f435b2fb-d990-4a6f-b29a-6abfe42ceef0" containerName="istio-proxy" Jul 10 02:55:00.779921 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.779902 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.783078 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.783049 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-37e4cb-openshift-default-dockercfg-n479j\"" Jul 10 02:55:00.783578 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.783553 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:00.838210 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w2jf\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838210 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838443 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838443 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838357 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838443 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838443 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838769 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838769 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.838769 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.838538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939348 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2jf\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939348 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939348 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.939585 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.940167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.939867 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.940167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.940101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.940167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.940133 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.940370 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.940163 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.940596 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.940573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.941866 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.941844 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.942103 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.942073 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.947024 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.947000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:00.947352 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:00.947324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w2jf\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf\") pod \"e2e-api-b-37e4cb-openshift-default-646677694f-tllzd\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:01.093215 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.093140 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:01.233368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.233308 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:01.235695 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:55:01.235669 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod679570af_4072_411b_a6ce_6eacf86ff955.slice/crio-a40d30e4ba299ea7c825e4f115dcd60955d13bdd61d8a3d025c8cf3acf470816 WatchSource:0}: Error finding container a40d30e4ba299ea7c825e4f115dcd60955d13bdd61d8a3d025c8cf3acf470816: Status 404 returned error can't find the container with id a40d30e4ba299ea7c825e4f115dcd60955d13bdd61d8a3d025c8cf3acf470816 Jul 10 02:55:01.237631 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.237601 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:01.237703 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.237662 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:01.237758 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.237712 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:01.704690 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.704655 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerStarted","Data":"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3"} Jul 10 02:55:01.704861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.704715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerStarted","Data":"a40d30e4ba299ea7c825e4f115dcd60955d13bdd61d8a3d025c8cf3acf470816"} Jul 10 02:55:01.726506 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:01.726455 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podStartSLOduration=1.726434304 podStartE2EDuration="1.726434304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:55:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:55:01.726004716 +0000 UTC m=+1929.002794163" watchObservedRunningTime="2026-07-10 02:55:01.726434304 +0000 UTC m=+1929.003223778" Jul 10 02:55:02.093825 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:02.093784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:02.098819 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:02.098791 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:02.710415 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:02.710388 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:02.711135 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:02.711117 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:11.713410 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:11.713367 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 02:55:11.713787 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:11.713420 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 02:55:12.769409 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.769376 2560 generic.go:356] "Generic (PLEG): container finished" podID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerID="3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4" exitCode=0 Jul 10 02:55:12.769834 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.769446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerDied","Data":"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4"} Jul 10 02:55:12.769834 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.769806 2560 scope.go:117] "RemoveContainer" containerID="3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4" Jul 10 02:55:12.770760 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.770736 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:12.770826 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.770804 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:12.770864 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:12.770836 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:13.778199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:13.778166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerStarted","Data":"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d"} Jul 10 02:55:13.854484 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:13.854445 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:13.854770 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:13.854738 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 02:55:13.854993 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:13.854965 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 02:55:14.841543 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:14.841064 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 02:55:14.841543 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:14.841122 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 02:55:15.841410 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:15.841370 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 02:55:15.841612 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:15.841439 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 02:55:16.844678 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:16.844639 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 02:55:16.845317 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:16.844708 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 02:55:17.712082 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.712048 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:17.712318 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.712099 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:17.804806 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.804772 2560 generic.go:356] "Generic (PLEG): container finished" podID="679570af-4072-411b-a6ce-6eacf86ff955" containerID="00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3" exitCode=0 Jul 10 02:55:17.804969 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.804843 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerDied","Data":"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3"} Jul 10 02:55:17.805257 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.805239 2560 scope.go:117] "RemoveContainer" containerID="00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3" Jul 10 02:55:17.806240 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.806214 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:17.806349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.806331 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:17.806401 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.806387 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:17.854167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.843622 2560 patch_prober.go:28] interesting pod/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 02:55:17.854167 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:17.843668 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 02:55:18.812896 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:18.812862 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerStarted","Data":"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b"} Jul 10 02:55:18.897940 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:18.897899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:18.898451 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:18.898430 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:18.899328 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:18.899303 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:19.094120 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:19.094028 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:19.094406 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:19.094380 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:19.094463 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:19.094445 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:20.094499 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:20.094467 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:20.094905 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:20.094531 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:21.093963 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:21.093929 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:21.094123 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:21.093980 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:22.094461 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:22.094425 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:22.094861 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:22.094490 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:22.388754 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:22.388671 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:55:22.388962 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:22.388935 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" containerID="cri-o://89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d" gracePeriod=30 Jul 10 02:55:23.094031 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:23.093988 2560 patch_prober.go:28] interesting pod/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 10 02:55:23.094248 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:23.094215 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 10 02:55:24.116292 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:24.116238 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:24.117289 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:24.117234 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:24.118472 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:24.118451 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:27.637592 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.637569 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:27.688558 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688494 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688558 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688525 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688558 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688548 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688746 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688577 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688746 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688633 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688746 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688665 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688746 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688694 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688951 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688755 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688951 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688802 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxj6r\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r\") pod \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\" (UID: \"7c32f556-d5ca-465e-aa59-fdf6d15b6062\") " Jul 10 02:55:27.688951 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688820 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:27.688951 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.688904 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:27.689169 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.689071 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:27.689169 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.689114 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:55:27.689169 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.689132 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.689169 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.689155 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.689413 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.689319 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:27.690643 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.690620 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:55:27.690756 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.690715 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:27.690995 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.690980 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r" (OuterVolumeSpecName: "kube-api-access-pxj6r") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "kube-api-access-pxj6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:55:27.691064 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.691041 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c32f556-d5ca-465e-aa59-fdf6d15b6062" (UID: "7c32f556-d5ca-465e-aa59-fdf6d15b6062"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:55:27.790320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790299 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790320 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790317 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790456 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790325 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790456 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790334 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c32f556-d5ca-465e-aa59-fdf6d15b6062-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790456 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790342 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790456 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790351 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c32f556-d5ca-465e-aa59-fdf6d15b6062-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.790456 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.790359 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxj6r\" (UniqueName: \"kubernetes.io/projected/7c32f556-d5ca-465e-aa59-fdf6d15b6062-kube-api-access-pxj6r\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:27.856339 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.856310 2560 generic.go:356] "Generic (PLEG): container finished" podID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerID="89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d" exitCode=0 Jul 10 02:55:27.856435 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.856366 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" Jul 10 02:55:27.856435 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.856401 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerDied","Data":"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d"} Jul 10 02:55:27.856515 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.856453 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q" event={"ID":"7c32f556-d5ca-465e-aa59-fdf6d15b6062","Type":"ContainerDied","Data":"ab0823f8943cd66fde906295061b72894c1be086bf3fe170a57ae2b23e00ac9c"} Jul 10 02:55:27.856515 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.856474 2560 scope.go:117] "RemoveContainer" containerID="89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d" Jul 10 02:55:27.870882 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.870865 2560 scope.go:117] "RemoveContainer" containerID="3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4" Jul 10 02:55:27.878681 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.878659 2560 scope.go:117] "RemoveContainer" containerID="89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d" Jul 10 02:55:27.879213 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:55:27.879003 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d\": container with ID starting with 89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d not found: ID does not exist" containerID="89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d" Jul 10 02:55:27.879213 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.879036 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d"} err="failed to get container status \"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d\": rpc error: code = NotFound desc = could not find container \"89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d\": container with ID starting with 89be46083fbcdc6498af75f5e4195d930dac1a4862702bde1b9a6c0a8a0cd54d not found: ID does not exist" Jul 10 02:55:27.879213 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.879058 2560 scope.go:117] "RemoveContainer" containerID="3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4" Jul 10 02:55:27.879540 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:55:27.879395 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4\": container with ID starting with 3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4 not found: ID does not exist" containerID="3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4" Jul 10 02:55:27.879540 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.879420 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4"} err="failed to get container status \"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4\": rpc error: code = NotFound desc = could not find container \"3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4\": container with ID starting with 3668106584ff46e30e716794e5b3b50e7f83a30abdfe82308b4050a2582881c4 not found: ID does not exist" Jul 10 02:55:27.881142 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.881120 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:55:27.885194 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:27.885172 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6e451c-openshift-default-6b6d769587-5h64q"] Jul 10 02:55:29.224175 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:29.224145 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" path="/var/lib/kubelet/pods/7c32f556-d5ca-465e-aa59-fdf6d15b6062/volumes" Jul 10 02:55:29.318727 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:29.318702 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:29.318899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:29.318879 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" containerID="cri-o://ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b" gracePeriod=30 Jul 10 02:55:30.075011 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.074973 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 02:55:30.075345 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.075333 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:30.075395 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.075347 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:30.075395 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.075355 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:30.075395 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.075364 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:30.075491 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.075436 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:30.079889 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.079869 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.082298 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.082262 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1d77d1-openshift-default-dockercfg-mlp9h\"" Jul 10 02:55:30.092982 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.092961 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 02:55:30.111478 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.111621 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111504 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.111621 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.111808 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.111808 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111695 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.111808 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbcnc\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.112261 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.112261 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.112261 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.111995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.212996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.212968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213199 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213184 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213541 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213541 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbcnc\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213541 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213713 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213713 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213617 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.213713 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213669 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.214008 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.213983 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.215629 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.215606 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.215720 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.215641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.221311 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.221292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbcnc\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.221406 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.221366 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token\") pod \"e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.389803 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.389710 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:30.520671 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.520647 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 02:55:30.522425 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:55:30.522404 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b5c9e5_7965_4d63_a3d0_0bf22989efc3.slice/crio-91ce8b979b9b59c619826fa6b0822387128358e646632300701495e6e106642b WatchSource:0}: Error finding container 91ce8b979b9b59c619826fa6b0822387128358e646632300701495e6e106642b: Status 404 returned error can't find the container with id 91ce8b979b9b59c619826fa6b0822387128358e646632300701495e6e106642b Jul 10 02:55:30.524582 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.524550 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:30.524653 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.524619 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:30.524653 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.524649 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:30.873865 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.873831 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerStarted","Data":"72a226efc3bad2cb91dca8a58c6af7005005505af99488dc6d02b4d39b2c8dcc"} Jul 10 02:55:30.874105 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.873888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerStarted","Data":"91ce8b979b9b59c619826fa6b0822387128358e646632300701495e6e106642b"} Jul 10 02:55:30.894019 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:30.893973 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podStartSLOduration=0.89395746 podStartE2EDuration="893.95746ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:55:30.891646578 +0000 UTC m=+1958.168436028" watchObservedRunningTime="2026-07-10 02:55:30.89395746 +0000 UTC m=+1958.170746905" Jul 10 02:55:31.390149 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:31.390123 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:31.395023 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:31.395002 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:31.878129 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:31.878098 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:31.879073 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:31.879055 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:55:34.561487 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.561466 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:34.651851 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651781 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.651851 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651821 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.651851 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651840 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651871 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651895 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651940 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.651971 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652029 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652062 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w2jf\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf\") pod \"679570af-4072-411b-a6ce-6eacf86ff955\" (UID: \"679570af-4072-411b-a6ce-6eacf86ff955\") " Jul 10 02:55:34.652094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652081 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:34.652470 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652090 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data" (OuterVolumeSpecName: "istio-data") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:34.652470 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652311 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:34.652470 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652329 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:34.652470 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652375 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.652470 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652389 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.652713 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.652599 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:55:34.654053 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.654023 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token" (OuterVolumeSpecName: "istio-token") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:55:34.654165 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.654076 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:55:34.654165 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.654125 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 02:55:34.654289 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.654210 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf" (OuterVolumeSpecName: "kube-api-access-6w2jf") pod "679570af-4072-411b-a6ce-6eacf86ff955" (UID: "679570af-4072-411b-a6ce-6eacf86ff955"). InnerVolumeSpecName "kube-api-access-6w2jf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:55:34.753094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753063 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753094 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753090 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w2jf\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-kube-api-access-6w2jf\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753103 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/679570af-4072-411b-a6ce-6eacf86ff955-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753115 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/679570af-4072-411b-a6ce-6eacf86ff955-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753124 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/679570af-4072-411b-a6ce-6eacf86ff955-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753136 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.753349 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.753146 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/679570af-4072-411b-a6ce-6eacf86ff955-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 02:55:34.891908 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.891880 2560 generic.go:356] "Generic (PLEG): container finished" podID="679570af-4072-411b-a6ce-6eacf86ff955" containerID="ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b" exitCode=0 Jul 10 02:55:34.892019 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.891929 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" Jul 10 02:55:34.892019 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.891955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerDied","Data":"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b"} Jul 10 02:55:34.892019 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.892003 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd" event={"ID":"679570af-4072-411b-a6ce-6eacf86ff955","Type":"ContainerDied","Data":"a40d30e4ba299ea7c825e4f115dcd60955d13bdd61d8a3d025c8cf3acf470816"} Jul 10 02:55:34.892137 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.892026 2560 scope.go:117] "RemoveContainer" containerID="ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b" Jul 10 02:55:34.901768 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.901749 2560 scope.go:117] "RemoveContainer" containerID="00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3" Jul 10 02:55:34.909202 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.909185 2560 scope.go:117] "RemoveContainer" containerID="ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b" Jul 10 02:55:34.909456 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:55:34.909440 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b\": container with ID starting with ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b not found: ID does not exist" containerID="ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b" Jul 10 02:55:34.909526 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.909463 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b"} err="failed to get container status \"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b\": rpc error: code = NotFound desc = could not find container \"ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b\": container with ID starting with ceefa264a498ed494a187325a02de7b993281ae9031887b5bf7859474cd4d38b not found: ID does not exist" Jul 10 02:55:34.909526 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.909479 2560 scope.go:117] "RemoveContainer" containerID="00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3" Jul 10 02:55:34.909683 ip-10-0-133-192 kubenswrapper[2560]: E0710 02:55:34.909665 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3\": container with ID starting with 00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3 not found: ID does not exist" containerID="00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3" Jul 10 02:55:34.909725 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.909689 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3"} err="failed to get container status \"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3\": rpc error: code = NotFound desc = could not find container \"00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3\": container with ID starting with 00005fe058432aceba92aae788a95597ef6c80b7267c9bbd1f96627e60f14fc3 not found: ID does not exist" Jul 10 02:55:34.924417 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.924392 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:34.940312 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:34.940288 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-37e4cb-openshift-default-646677694f-tllzd"] Jul 10 02:55:35.224160 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:35.224089 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="679570af-4072-411b-a6ce-6eacf86ff955" path="/var/lib/kubelet/pods/679570af-4072-411b-a6ce-6eacf86ff955/volumes" Jul 10 02:55:37.805630 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.805593 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 02:55:37.805996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.805942 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.805996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.805953 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.805996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.805970 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.805996 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.805976 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.806130 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.806045 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c32f556-d5ca-465e-aa59-fdf6d15b6062" containerName="istio-proxy" Jul 10 02:55:37.806130 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.806053 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.806130 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.806064 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="679570af-4072-411b-a6ce-6eacf86ff955" containerName="istio-proxy" Jul 10 02:55:37.810597 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.810568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.812890 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.812871 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f94570-openshift-default-dockercfg-qrr45\"" Jul 10 02:55:37.823894 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.823864 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 02:55:37.878127 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878104 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878238 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878240 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5twd\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878368 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.878489 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.878392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979261 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5twd\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979337 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979440 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979684 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979666 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979840 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979899 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979854 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.979959 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.979925 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.980057 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.980037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.980112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.980066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.980112 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.980074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.981723 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.981702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.981964 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.981946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.986725 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.986702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:37.986835 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:37.986765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5twd\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd\") pod \"e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:38.121852 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.121783 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:38.256528 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.256505 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 02:55:38.258893 ip-10-0-133-192 kubenswrapper[2560]: W0710 02:55:38.258866 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5ff268_834f_45c0_b0c2_5ef5cb9ee987.slice/crio-80dcd0fad905bfeb32cdac5bf40cd5b6c70fcd2e4fb2ad12047c37e32de5453c WatchSource:0}: Error finding container 80dcd0fad905bfeb32cdac5bf40cd5b6c70fcd2e4fb2ad12047c37e32de5453c: Status 404 returned error can't find the container with id 80dcd0fad905bfeb32cdac5bf40cd5b6c70fcd2e4fb2ad12047c37e32de5453c Jul 10 02:55:38.261401 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.261369 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:38.261484 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.261435 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:38.261484 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.261462 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:38.911319 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.911279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerStarted","Data":"2c0b5efaa113362d260c86d83cb3eefce62a905a1dd83297a355aa588252e3c5"} Jul 10 02:55:38.911679 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.911334 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerStarted","Data":"80dcd0fad905bfeb32cdac5bf40cd5b6c70fcd2e4fb2ad12047c37e32de5453c"} Jul 10 02:55:38.932085 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:38.932040 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podStartSLOduration=1.932025892 podStartE2EDuration="1.932025892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:55:38.930398366 +0000 UTC m=+1966.207187814" watchObservedRunningTime="2026-07-10 02:55:38.932025892 +0000 UTC m=+1966.208815338" Jul 10 02:55:39.122323 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:39.122284 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:39.126561 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:39.126538 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:39.915698 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:39.915667 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:39.916664 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:39.916648 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:54.916658 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:54.916626 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:55:54.917058 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:54.916687 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:55:54.999443 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:54.999413 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerID="2c0b5efaa113362d260c86d83cb3eefce62a905a1dd83297a355aa588252e3c5" exitCode=0 Jul 10 02:55:54.999572 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:54.999488 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerDied","Data":"2c0b5efaa113362d260c86d83cb3eefce62a905a1dd83297a355aa588252e3c5"} Jul 10 02:55:54.999863 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:54.999849 2560 scope.go:117] "RemoveContainer" containerID="2c0b5efaa113362d260c86d83cb3eefce62a905a1dd83297a355aa588252e3c5" Jul 10 02:55:55.000841 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:55.000814 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:55.000894 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:55.000883 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:55.000929 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:55.000916 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:55:56.006247 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:56.006218 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerStarted","Data":"3c50cac64174f5032a7352322f0c93a06d15002cd87c469c73ffed4eca21564b"} Jul 10 02:55:56.122680 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:56.122647 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:55:56.122988 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:56.122969 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:55:56.123034 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:56.123010 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:55:57.122602 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:57.122562 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:55:57.123067 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:57.122625 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:55:58.123156 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:58.123121 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:55:58.123590 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:58.123175 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:55:59.122312 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:59.122253 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:55:59.122503 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:55:59.122329 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:56:00.123211 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:00.123173 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:56:00.123672 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:00.123237 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:56:01.122163 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:01.122128 2560 patch_prober.go:28] interesting pod/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 02:56:01.122357 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:01.122184 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 02:56:02.126545 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:02.126517 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:56:02.126994 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:02.126817 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:56:02.127569 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:02.127548 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 02:56:13.100010 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.099979 2560 generic.go:356] "Generic (PLEG): container finished" podID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerID="72a226efc3bad2cb91dca8a58c6af7005005505af99488dc6d02b4d39b2c8dcc" exitCode=0 Jul 10 02:56:13.100449 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.100027 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerDied","Data":"72a226efc3bad2cb91dca8a58c6af7005005505af99488dc6d02b4d39b2c8dcc"} Jul 10 02:56:13.100449 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.100370 2560 scope.go:117] "RemoveContainer" containerID="72a226efc3bad2cb91dca8a58c6af7005005505af99488dc6d02b4d39b2c8dcc" Jul 10 02:56:13.101308 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.101279 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:56:13.101418 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.101372 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:56:13.101482 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:13.101433 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 02:56:14.107162 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:14.107126 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerStarted","Data":"47655bf9876c78f6ab20a98a5546c8964b8b3e2b781829fd05e89e0414f30e10"} Jul 10 02:56:14.390104 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:14.390013 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:56:14.390978 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:14.390956 2560 patch_prober.go:28] interesting pod/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 10 02:56:14.391039 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:14.391004 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 10 02:56:15.390545 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:15.390513 2560 patch_prober.go:28] interesting pod/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 10 02:56:15.390943 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:15.390567 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 10 02:56:16.390032 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:16.389994 2560 patch_prober.go:28] interesting pod/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 10 02:56:16.390230 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:16.390066 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 10 02:56:16.878441 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:16.878408 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:56:17.390250 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:17.390212 2560 patch_prober.go:28] interesting pod/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 10 02:56:17.390459 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:17.390323 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 10 02:56:18.391061 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:18.391027 2560 patch_prober.go:28] interesting pod/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 10 02:56:18.391490 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:18.391108 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 10 02:56:19.394109 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:19.394080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:56:19.395017 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:56:19.394999 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 02:57:53.253924 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:57:53.253897 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 02:57:53.258870 ip-10-0-133-192 kubenswrapper[2560]: I0710 02:57:53.258849 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 03:00:00.203411 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.203377 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:00:00.205542 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.205525 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:00:00.205651 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.205633 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" Jul 10 03:00:00.207439 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.207422 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:00:00.207540 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.207518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" Jul 10 03:00:00.209094 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.209079 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" Jul 10 03:00:00.209799 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.209780 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6chdf\"" Jul 10 03:00:00.216229 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.216211 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" Jul 10 03:00:00.223744 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.223724 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:00:00.225545 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.225525 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" Jul 10 03:00:00.226419 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.226400 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:00:00.235014 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.234942 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:00:00.260129 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.260107 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" Jul 10 03:00:00.399470 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.399443 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:00:00.401606 ip-10-0-133-192 kubenswrapper[2560]: W0710 03:00:00.401568 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53632c2c_2547_4573_aa79_fa487a2626dc.slice/crio-100a935a4a7c66d861cb597695ae23b5a0cca324f7e14dd1ac92870b70d49e96 WatchSource:0}: Error finding container 100a935a4a7c66d861cb597695ae23b5a0cca324f7e14dd1ac92870b70d49e96: Status 404 returned error can't find the container with id 100a935a4a7c66d861cb597695ae23b5a0cca324f7e14dd1ac92870b70d49e96 Jul 10 03:00:00.449547 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.449518 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:00:00.456472 ip-10-0-133-192 kubenswrapper[2560]: W0710 03:00:00.456447 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1000f4_639d_4e28_aaaf_0b9e6407a6c3.slice/crio-a339a232448090f2438da5ea1c6dc0fab71f586f1aef93f521f13f321dfd3611 WatchSource:0}: Error finding container a339a232448090f2438da5ea1c6dc0fab71f586f1aef93f521f13f321dfd3611: Status 404 returned error can't find the container with id a339a232448090f2438da5ea1c6dc0fab71f586f1aef93f521f13f321dfd3611 Jul 10 03:00:00.610905 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:00.610877 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:00:00.613114 ip-10-0-133-192 kubenswrapper[2560]: W0710 03:00:00.613089 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896a5d69_ee73_4ac8_8f16_51d926741ed2.slice/crio-b70a6137eaba991c1a0e1a12b78f7600e7dcde182dcaf0b4ee551fa550757dae WatchSource:0}: Error finding container b70a6137eaba991c1a0e1a12b78f7600e7dcde182dcaf0b4ee551fa550757dae: Status 404 returned error can't find the container with id b70a6137eaba991c1a0e1a12b78f7600e7dcde182dcaf0b4ee551fa550757dae Jul 10 03:00:01.101139 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.101102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" event={"ID":"896a5d69-ee73-4ac8-8f16-51d926741ed2","Type":"ContainerStarted","Data":"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d"} Jul 10 03:00:01.101355 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.101184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" event={"ID":"896a5d69-ee73-4ac8-8f16-51d926741ed2","Type":"ContainerStarted","Data":"b70a6137eaba991c1a0e1a12b78f7600e7dcde182dcaf0b4ee551fa550757dae"} Jul 10 03:00:01.102629 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.102599 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" event={"ID":"53632c2c-2547-4573-aa79-fa487a2626dc","Type":"ContainerStarted","Data":"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14"} Jul 10 03:00:01.102765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.102641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" event={"ID":"53632c2c-2547-4573-aa79-fa487a2626dc","Type":"ContainerStarted","Data":"100a935a4a7c66d861cb597695ae23b5a0cca324f7e14dd1ac92870b70d49e96"} Jul 10 03:00:01.103923 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.103900 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" event={"ID":"de1000f4-639d-4e28-aaaf-0b9e6407a6c3","Type":"ContainerStarted","Data":"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9"} Jul 10 03:00:01.104025 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.103938 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" event={"ID":"de1000f4-639d-4e28-aaaf-0b9e6407a6c3","Type":"ContainerStarted","Data":"a339a232448090f2438da5ea1c6dc0fab71f586f1aef93f521f13f321dfd3611"} Jul 10 03:00:01.117770 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.117716 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" podStartSLOduration=1.117703199 podStartE2EDuration="1.117703199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:00:01.116792048 +0000 UTC m=+2228.393581515" watchObservedRunningTime="2026-07-10 03:00:01.117703199 +0000 UTC m=+2228.394492644" Jul 10 03:00:01.131317 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.131251 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" podStartSLOduration=1.131236965 podStartE2EDuration="1.131236965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:00:01.13123091 +0000 UTC m=+2228.408020383" watchObservedRunningTime="2026-07-10 03:00:01.131236965 +0000 UTC m=+2228.408026418" Jul 10 03:00:01.146867 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:00:01.146835 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" podStartSLOduration=1.146823365 podStartE2EDuration="1.146823365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:00:01.146443199 +0000 UTC m=+2228.423232646" watchObservedRunningTime="2026-07-10 03:00:01.146823365 +0000 UTC m=+2228.423612862" Jul 10 03:01:00.011395 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.011363 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:01:00.011898 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.011577 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" podUID="53632c2c-2547-4573-aa79-fa487a2626dc" containerName="cleanup" containerID="cri-o://42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14" gracePeriod=30 Jul 10 03:01:00.015937 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.015909 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:01:00.016154 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.016116 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" podUID="896a5d69-ee73-4ac8-8f16-51d926741ed2" containerName="cleanup" containerID="cri-o://d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d" gracePeriod=30 Jul 10 03:01:00.017436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.017410 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:01:00.017631 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:00.017589 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" podUID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" containerName="cleanup" containerID="cri-o://9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9" gracePeriod=30 Jul 10 03:01:13.272782 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:13.272745 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 03:01:13.274079 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:13.272944 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" containerID="cri-o://47655bf9876c78f6ab20a98a5546c8964b8b3e2b781829fd05e89e0414f30e10" gracePeriod=30 Jul 10 03:01:18.457658 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.457625 2560 generic.go:356] "Generic (PLEG): container finished" podID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerID="47655bf9876c78f6ab20a98a5546c8964b8b3e2b781829fd05e89e0414f30e10" exitCode=0 Jul 10 03:01:18.458112 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.457705 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerDied","Data":"47655bf9876c78f6ab20a98a5546c8964b8b3e2b781829fd05e89e0414f30e10"} Jul 10 03:01:18.458112 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.457769 2560 scope.go:117] "RemoveContainer" containerID="72a226efc3bad2cb91dca8a58c6af7005005505af99488dc6d02b4d39b2c8dcc" Jul 10 03:01:18.521920 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.521883 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 03:01:18.641511 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641442 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641511 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641477 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641525 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641636 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbcnc\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641738 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641775 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641800 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:18.641873 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641840 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.642218 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641881 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.642218 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641899 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket\") pod \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\" (UID: \"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3\") " Jul 10 03:01:18.642218 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.641934 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:01:18.642218 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642049 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:18.642436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642227 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.642436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642246 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.642436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642258 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.642436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642279 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:18.642436 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.642321 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:18.643805 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.643782 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:01:18.643893 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.643805 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc" (OuterVolumeSpecName: "kube-api-access-tbcnc") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "kube-api-access-tbcnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:01:18.644018 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.644000 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 03:01:18.644167 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.644153 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" (UID: "c0b5c9e5-7965-4d63-a3d0-0bf22989efc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:18.742972 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.742940 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbcnc\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-kube-api-access-tbcnc\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.742972 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.742973 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.742972 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.742982 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.743141 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.742991 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.743141 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.743000 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:18.743141 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:18.743008 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:19.463629 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:19.463605 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" Jul 10 03:01:19.464019 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:19.463603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d" event={"ID":"c0b5c9e5-7965-4d63-a3d0-0bf22989efc3","Type":"ContainerDied","Data":"91ce8b979b9b59c619826fa6b0822387128358e646632300701495e6e106642b"} Jul 10 03:01:19.464019 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:19.463733 2560 scope.go:117] "RemoveContainer" containerID="47655bf9876c78f6ab20a98a5546c8964b8b3e2b781829fd05e89e0414f30e10" Jul 10 03:01:19.480754 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:19.480733 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 03:01:19.485026 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:19.484972 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1d77d1-openshift-default-7fb5757775-csx7d"] Jul 10 03:01:20.305413 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:20.305373 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 03:01:20.305719 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:20.305687 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" containerID="cri-o://3c50cac64174f5032a7352322f0c93a06d15002cd87c469c73ffed4eca21564b" gracePeriod=30 Jul 10 03:01:21.224474 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:21.224443 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" path="/var/lib/kubelet/pods/c0b5c9e5-7965-4d63-a3d0-0bf22989efc3/volumes" Jul 10 03:01:25.494801 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.494772 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerID="3c50cac64174f5032a7352322f0c93a06d15002cd87c469c73ffed4eca21564b" exitCode=0 Jul 10 03:01:25.495213 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.494857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerDied","Data":"3c50cac64174f5032a7352322f0c93a06d15002cd87c469c73ffed4eca21564b"} Jul 10 03:01:25.495213 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.494936 2560 scope.go:117] "RemoveContainer" containerID="2c0b5efaa113362d260c86d83cb3eefce62a905a1dd83297a355aa588252e3c5" Jul 10 03:01:25.557801 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.557779 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 03:01:25.698170 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698100 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698170 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698135 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698177 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698194 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698214 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698309 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698337 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698363 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5twd\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698370 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:25.698399 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698391 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data\") pod \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\" (UID: \"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987\") " Jul 10 03:01:25.698791 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698562 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:25.698791 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698601 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:25.698791 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698621 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:01:25.698791 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698744 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:25.699005 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698805 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istiod-ca-cert\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.699005 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698819 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.699005 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698839 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-workload-certs\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.699005 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.698848 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-credential-socket\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.700533 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.700506 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:25.700640 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.700554 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 03:01:25.700867 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.700833 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd" (OuterVolumeSpecName: "kube-api-access-s5twd") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "kube-api-access-s5twd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:01:25.700958 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.700846 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" (UID: "cc5ff268-834f-45c0-b0c2-5ef5cb9ee987"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:01:25.800088 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.800065 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-podinfo\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.800088 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.800086 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5twd\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-kube-api-access-s5twd\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.800208 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.800096 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-data\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.800208 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.800104 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-token\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:25.800208 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:25.800111 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987-istio-envoy\") on node \"ip-10-0-133-192.ec2.internal\" DevicePath \"\"" Jul 10 03:01:26.414971 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.414942 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc46458b-plz8t_72cbaf8e-0475-476e-b827-19213ced8b66/copy-manifests/0.log" Jul 10 03:01:26.421986 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.421962 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc46458b-plz8t_72cbaf8e-0475-476e-b827-19213ced8b66/manager/0.log" Jul 10 03:01:26.501254 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.501235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" event={"ID":"cc5ff268-834f-45c0-b0c2-5ef5cb9ee987","Type":"ContainerDied","Data":"80dcd0fad905bfeb32cdac5bf40cd5b6c70fcd2e4fb2ad12047c37e32de5453c"} Jul 10 03:01:26.501590 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.501252 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq" Jul 10 03:01:26.501590 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.501309 2560 scope.go:117] "RemoveContainer" containerID="3c50cac64174f5032a7352322f0c93a06d15002cd87c469c73ffed4eca21564b" Jul 10 03:01:26.523850 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.523830 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 03:01:26.527367 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.527347 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f94570-openshift-default-54c7c78f47-dqkkq"] Jul 10 03:01:26.981965 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:26.981938 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29727540-brpzn_896a5d69-ee73-4ac8-8f16-51d926741ed2/cleanup/0.log" Jul 10 03:01:27.087853 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:27.087823 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j_de1000f4-639d-4e28-aaaf-0b9e6407a6c3/cleanup/0.log" Jul 10 03:01:27.193061 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:27.193032 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr_53632c2c-2547-4573-aa79-fa487a2626dc/cleanup/0.log" Jul 10 03:01:27.224451 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:27.224421 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" path="/var/lib/kubelet/pods/cc5ff268-834f-45c0-b0c2-5ef5cb9ee987/volumes" Jul 10 03:01:29.075030 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:29.074996 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv7m5_fdb70edb-29b0-4e9b-a083-cac4dd20f71e/istio-proxy/0.log" Jul 10 03:01:29.194240 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:29.194202 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gxvz4_034a48a0-d150-4fb0-83f1-958973e7d240/discovery/0.log" Jul 10 03:01:29.531148 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:29.531119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/1.log" Jul 10 03:01:29.750056 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:29.750022 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-25htw_6c32bd56-9cac-45c8-9a58-9b96b36e19e6/payload-processing/0.log" Jul 10 03:01:30.226930 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.226908 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" Jul 10 03:01:30.250521 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.250495 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" Jul 10 03:01:30.253772 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.253741 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" Jul 10 03:01:30.295441 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.295381 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-z6qxz_fe765ea0-799d-47c2-bd14-4b0327dfb386/main/0.log" Jul 10 03:01:30.520546 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.520518 2560 generic.go:356] "Generic (PLEG): container finished" podID="896a5d69-ee73-4ac8-8f16-51d926741ed2" containerID="d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d" exitCode=137 Jul 10 03:01:30.520685 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.520560 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" Jul 10 03:01:30.520685 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.520584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" event={"ID":"896a5d69-ee73-4ac8-8f16-51d926741ed2","Type":"ContainerDied","Data":"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d"} Jul 10 03:01:30.520685 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.520635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn" event={"ID":"896a5d69-ee73-4ac8-8f16-51d926741ed2","Type":"ContainerDied","Data":"b70a6137eaba991c1a0e1a12b78f7600e7dcde182dcaf0b4ee551fa550757dae"} Jul 10 03:01:30.520685 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.520658 2560 scope.go:117] "RemoveContainer" containerID="d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d" Jul 10 03:01:30.521963 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.521940 2560 generic.go:356] "Generic (PLEG): container finished" podID="53632c2c-2547-4573-aa79-fa487a2626dc" containerID="42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14" exitCode=137 Jul 10 03:01:30.522057 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.521979 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" Jul 10 03:01:30.522057 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.522021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" event={"ID":"53632c2c-2547-4573-aa79-fa487a2626dc","Type":"ContainerDied","Data":"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14"} Jul 10 03:01:30.522161 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.522062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr" event={"ID":"53632c2c-2547-4573-aa79-fa487a2626dc","Type":"ContainerDied","Data":"100a935a4a7c66d861cb597695ae23b5a0cca324f7e14dd1ac92870b70d49e96"} Jul 10 03:01:30.523311 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.523292 2560 generic.go:356] "Generic (PLEG): container finished" podID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" containerID="9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9" exitCode=137 Jul 10 03:01:30.523419 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.523337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" event={"ID":"de1000f4-639d-4e28-aaaf-0b9e6407a6c3","Type":"ContainerDied","Data":"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9"} Jul 10 03:01:30.523419 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.523340 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" Jul 10 03:01:30.523419 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.523377 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j" event={"ID":"de1000f4-639d-4e28-aaaf-0b9e6407a6c3","Type":"ContainerDied","Data":"a339a232448090f2438da5ea1c6dc0fab71f586f1aef93f521f13f321dfd3611"} Jul 10 03:01:30.530543 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.530525 2560 scope.go:117] "RemoveContainer" containerID="d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d" Jul 10 03:01:30.530901 ip-10-0-133-192 kubenswrapper[2560]: E0710 03:01:30.530879 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d\": container with ID starting with d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d not found: ID does not exist" containerID="d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d" Jul 10 03:01:30.530985 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.530911 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d"} err="failed to get container status \"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d\": rpc error: code = NotFound desc = could not find container \"d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d\": container with ID starting with d1861cc810807072179ccdcefdf6ef01abc98ed8a82d2dc25e656074a4fbe30d not found: ID does not exist" Jul 10 03:01:30.530985 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.530933 2560 scope.go:117] "RemoveContainer" containerID="42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14" Jul 10 03:01:30.538621 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.538605 2560 scope.go:117] "RemoveContainer" containerID="42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14" Jul 10 03:01:30.538867 ip-10-0-133-192 kubenswrapper[2560]: E0710 03:01:30.538847 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14\": container with ID starting with 42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14 not found: ID does not exist" containerID="42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14" Jul 10 03:01:30.538915 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.538873 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14"} err="failed to get container status \"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14\": rpc error: code = NotFound desc = could not find container \"42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14\": container with ID starting with 42dab4a89fe2629519b26f7113938fac8e109dcda83e2afec1728f62b464ff14 not found: ID does not exist" Jul 10 03:01:30.538915 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.538889 2560 scope.go:117] "RemoveContainer" containerID="9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9" Jul 10 03:01:30.543494 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.543477 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:01:30.545139 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.545120 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727540-brpzn"] Jul 10 03:01:30.548185 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.548167 2560 scope.go:117] "RemoveContainer" containerID="9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9" Jul 10 03:01:30.548430 ip-10-0-133-192 kubenswrapper[2560]: E0710 03:01:30.548414 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9\": container with ID starting with 9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9 not found: ID does not exist" containerID="9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9" Jul 10 03:01:30.548484 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.548435 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9"} err="failed to get container status \"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9\": rpc error: code = NotFound desc = could not find container \"9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9\": container with ID starting with 9642185a3964b988df5d0d15954f890cd368d51d4f36ab456556e7c1089739e9 not found: ID does not exist" Jul 10 03:01:30.556904 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.555625 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:01:30.561485 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.561461 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1d77d1-29727540-kf99j"] Jul 10 03:01:30.571381 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.571357 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:01:30.575035 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:30.575018 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-f94570-29727540-jk5fr"] Jul 10 03:01:31.223692 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:31.223662 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53632c2c-2547-4573-aa79-fa487a2626dc" path="/var/lib/kubelet/pods/53632c2c-2547-4573-aa79-fa487a2626dc/volumes" Jul 10 03:01:31.223900 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:31.223887 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896a5d69-ee73-4ac8-8f16-51d926741ed2" path="/var/lib/kubelet/pods/896a5d69-ee73-4ac8-8f16-51d926741ed2/volumes" Jul 10 03:01:31.224094 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:31.224083 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" path="/var/lib/kubelet/pods/de1000f4-639d-4e28-aaaf-0b9e6407a6c3/volumes" Jul 10 03:01:37.863196 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:37.863165 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gqsgv_79edebb3-eeb7-4fbe-b368-99e83af6b4d1/global-pull-secret-syncer/0.log" Jul 10 03:01:38.034748 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:38.034716 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d7787_aba6bc18-cab9-41dd-a432-32bfe37df3df/konnectivity-agent/0.log" Jul 10 03:01:38.135529 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:38.135450 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-192.ec2.internal_d95c7e23d0cc9fb766a175e924a681da/haproxy/0.log" Jul 10 03:01:44.778185 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.778147 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sdr7b_3e69e863-c10e-49a0-a3de-82f43c3a2b4c/kube-state-metrics/0.log" Jul 10 03:01:44.794741 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.794711 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sdr7b_3e69e863-c10e-49a0-a3de-82f43c3a2b4c/kube-rbac-proxy-main/0.log" Jul 10 03:01:44.815054 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.815034 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-sdr7b_3e69e863-c10e-49a0-a3de-82f43c3a2b4c/kube-rbac-proxy-self/0.log" Jul 10 03:01:44.920316 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.920289 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78cp7_b5774476-6750-4cab-b767-08748f4d4b3e/node-exporter/0.log" Jul 10 03:01:44.955791 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.955769 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78cp7_b5774476-6750-4cab-b767-08748f4d4b3e/kube-rbac-proxy/0.log" Jul 10 03:01:44.976811 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:44.976792 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78cp7_b5774476-6750-4cab-b767-08748f4d4b3e/init-textfile/0.log" Jul 10 03:01:45.710028 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.709952 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/thanos-query/0.log" Jul 10 03:01:45.735577 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.735555 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/kube-rbac-proxy-web/0.log" Jul 10 03:01:45.759957 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.759933 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/kube-rbac-proxy/0.log" Jul 10 03:01:45.782650 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.782600 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/prom-label-proxy/0.log" Jul 10 03:01:45.804197 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.804181 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/kube-rbac-proxy-rules/0.log" Jul 10 03:01:45.825890 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:45.825865 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6fcd6bd446-dhg47_d3b7a0af-1717-45dc-a515-644ff7561da1/kube-rbac-proxy-metrics/0.log" Jul 10 03:01:48.569829 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.569790 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9"] Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570216 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53632c2c-2547-4573-aa79-fa487a2626dc" containerName="cleanup" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570234 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53632c2c-2547-4573-aa79-fa487a2626dc" containerName="cleanup" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570263 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570287 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570297 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" containerName="cleanup" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570306 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" containerName="cleanup" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570325 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570333 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570344 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570353 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570362 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896a5d69-ee73-4ac8-8f16-51d926741ed2" containerName="cleanup" Jul 10 03:01:48.570377 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570370 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="896a5d69-ee73-4ac8-8f16-51d926741ed2" containerName="cleanup" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570443 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570454 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="de1000f4-639d-4e28-aaaf-0b9e6407a6c3" containerName="cleanup" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570463 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b5c9e5-7965-4d63-a3d0-0bf22989efc3" containerName="istio-proxy" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570468 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="896a5d69-ee73-4ac8-8f16-51d926741ed2" containerName="cleanup" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570475 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53632c2c-2547-4573-aa79-fa487a2626dc" containerName="cleanup" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570481 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.570765 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.570487 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5ff268-834f-45c0-b0c2-5ef5cb9ee987" containerName="istio-proxy" Jul 10 03:01:48.574688 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.574672 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.577120 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.577101 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z9m4\"/\"kube-root-ca.crt\"" Jul 10 03:01:48.578095 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.578070 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2z9m4\"/\"openshift-service-ca.crt\"" Jul 10 03:01:48.578182 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.578075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2z9m4\"/\"default-dockercfg-k6xcg\"" Jul 10 03:01:48.582625 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.582605 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9"] Jul 10 03:01:48.667600 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.667567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzx59\" (UniqueName: \"kubernetes.io/projected/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-kube-api-access-mzx59\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.667745 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.667655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-lib-modules\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.667801 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.667763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-podres\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.667864 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.667803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-sys\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.667864 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.667837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-proc\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.671404 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.671378 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-l6r57_1738b965-a6b9-472b-9daa-af498799ab65/download-server/0.log" Jul 10 03:01:48.768529 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-lib-modules\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768638 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-podres\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768638 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-sys\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768638 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-proc\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768638 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768635 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzx59\" (UniqueName: \"kubernetes.io/projected/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-kube-api-access-mzx59\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768821 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768667 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-sys\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768821 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768677 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-podres\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768821 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768668 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-lib-modules\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.768821 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.768705 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-proc\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.778146 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.778122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzx59\" (UniqueName: \"kubernetes.io/projected/9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38-kube-api-access-mzx59\") pod \"perf-node-gather-daemonset-rgbq9\" (UID: \"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38\") " pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:48.885515 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:48.885463 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:49.009899 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:49.009856 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9"] Jul 10 03:01:49.012003 ip-10-0-133-192 kubenswrapper[2560]: W0710 03:01:49.011979 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9c383fb6_2ef9_4bb8_a7b2_cfdb891a5a38.slice/crio-7e2036869f0577de71d112b1e52a535ad5ff62868eba3f17d133147637578b92 WatchSource:0}: Error finding container 7e2036869f0577de71d112b1e52a535ad5ff62868eba3f17d133147637578b92: Status 404 returned error can't find the container with id 7e2036869f0577de71d112b1e52a535ad5ff62868eba3f17d133147637578b92 Jul 10 03:01:49.610523 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:49.610484 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" event={"ID":"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38","Type":"ContainerStarted","Data":"9f7b3cef9e476f6b5808d0628f8a575b2d0dc360ef0059b1f222db3da870362f"} Jul 10 03:01:49.610938 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:49.610538 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" event={"ID":"9c383fb6-2ef9-4bb8-a7b2-cfdb891a5a38","Type":"ContainerStarted","Data":"7e2036869f0577de71d112b1e52a535ad5ff62868eba3f17d133147637578b92"} Jul 10 03:01:49.610938 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:49.610566 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:49.631575 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:49.631531 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" podStartSLOduration=1.631516107 podStartE2EDuration="1.631516107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:01:49.631207969 +0000 UTC m=+2336.907997430" watchObservedRunningTime="2026-07-10 03:01:49.631516107 +0000 UTC m=+2336.908305552" Jul 10 03:01:50.549334 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:50.549297 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8hz8_e3b7624f-c3b1-4041-b6a1-0202d4772de0/dns/0.log" Jul 10 03:01:50.570908 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:50.570872 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8hz8_e3b7624f-c3b1-4041-b6a1-0202d4772de0/kube-rbac-proxy/0.log" Jul 10 03:01:50.593318 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:50.593289 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7lpds_ab9baa9e-314f-4d93-aba3-0bc72da716c5/dns-node-resolver/0.log" Jul 10 03:01:51.190414 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:51.190383 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-749ffcb46b-cndhj_04716cff-18b9-4372-92d0-444f4b236c51/registry/0.log" Jul 10 03:01:51.268319 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:51.268296 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mszkn_6d4ef782-8911-480a-90dd-63d7de6699f3/node-ca/0.log" Jul 10 03:01:52.206069 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:52.206036 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv7m5_fdb70edb-29b0-4e9b-a083-cac4dd20f71e/istio-proxy/0.log" Jul 10 03:01:52.429158 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:52.429134 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gxvz4_034a48a0-d150-4fb0-83f1-958973e7d240/discovery/0.log" Jul 10 03:01:52.504063 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:52.503992 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/0.log" Jul 10 03:01:52.518312 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:52.518260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/1.log" Jul 10 03:01:52.566734 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:52.566715 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-25htw_6c32bd56-9cac-45c8-9a58-9b96b36e19e6/payload-processing/0.log" Jul 10 03:01:53.332996 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:53.332967 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tbkfz_375993c8-bcec-4457-b129-456c3fe3b9c2/serve-healthcheck-canary/0.log" Jul 10 03:01:53.880147 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:53.880120 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlp29_0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0/kube-rbac-proxy/0.log" Jul 10 03:01:53.903387 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:53.903357 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlp29_0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0/exporter/0.log" Jul 10 03:01:53.924150 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:53.924086 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlp29_0cc2ac12-ee8f-4492-bf5b-e0cf7f244fa0/extractor/0.log" Jul 10 03:01:55.624973 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:55.624949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2z9m4/perf-node-gather-daemonset-rgbq9" Jul 10 03:01:56.599781 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:56.599735 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc46458b-plz8t_72cbaf8e-0475-476e-b827-19213ced8b66/manager/0.log" Jul 10 03:01:56.624590 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:56.624567 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc46458b-plz8t_72cbaf8e-0475-476e-b827-19213ced8b66/copy-manifests/0.log" Jul 10 03:01:58.076993 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:58.076959 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5fcbd9fc8-sglnm_9d8d8706-d8ca-4e0f-89c4-b926a19ea2dd/manager/0.log" Jul 10 03:01:59.272589 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:59.272563 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv7m5_fdb70edb-29b0-4e9b-a083-cac4dd20f71e/istio-proxy/0.log" Jul 10 03:01:59.485760 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:59.485731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gxvz4_034a48a0-d150-4fb0-83f1-958973e7d240/discovery/0.log" Jul 10 03:01:59.536770 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:59.536679 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/0.log" Jul 10 03:01:59.549827 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:59.549795 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/1.log" Jul 10 03:01:59.581163 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:01:59.581142 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-25htw_6c32bd56-9cac-45c8-9a58-9b96b36e19e6/payload-processing/0.log" Jul 10 03:02:00.835426 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:00.835395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qv7m5_fdb70edb-29b0-4e9b-a083-cac4dd20f71e/istio-proxy/0.log" Jul 10 03:02:01.054136 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:01.054103 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gxvz4_034a48a0-d150-4fb0-83f1-958973e7d240/discovery/0.log" Jul 10 03:02:01.109548 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:01.109476 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/0.log" Jul 10 03:02:01.121949 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:01.121925 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-sw79h_b18920e3-8734-471d-b4db-a16129a305be/istio-proxy/1.log" Jul 10 03:02:01.150332 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:01.150306 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-25htw_6c32bd56-9cac-45c8-9a58-9b96b36e19e6/payload-processing/0.log" Jul 10 03:02:03.296549 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:03.296521 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-pqhbz_73d6d0fb-a0c8-4f23-bab3-806aca111d94/migrator/0.log" Jul 10 03:02:03.315934 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:03.315907 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-pqhbz_73d6d0fb-a0c8-4f23-bab3-806aca111d94/graceful-termination/0.log" Jul 10 03:02:03.667827 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:03.667717 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vbp4t_f4c954d2-f958-4402-8129-3d8d188b5d1e/kube-storage-version-migrator-operator/1.log" Jul 10 03:02:03.669624 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:03.669577 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vbp4t_f4c954d2-f958-4402-8129-3d8d188b5d1e/kube-storage-version-migrator-operator/0.log" Jul 10 03:02:04.689418 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:04.689336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-82xkg_6b50a459-0940-404b-b449-080620155638/kube-multus/0.log" Jul 10 03:02:05.027882 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.027856 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/kube-multus-additional-cni-plugins/0.log" Jul 10 03:02:05.049064 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.049045 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/egress-router-binary-copy/0.log" Jul 10 03:02:05.069220 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.069204 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/cni-plugins/0.log" Jul 10 03:02:05.089958 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.089940 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/bond-cni-plugin/0.log" Jul 10 03:02:05.110849 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.110828 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/routeoverride-cni/0.log" Jul 10 03:02:05.133182 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.133161 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/whereabouts-cni-bincopy/0.log" Jul 10 03:02:05.153597 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.153576 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v8sj7_34b111d8-1f9a-41a2-baf7-2ad06a1a04a9/whereabouts-cni/0.log" Jul 10 03:02:05.288899 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.288836 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j6wg4_179493af-371a-414e-9602-5de5cae4d7a8/network-metrics-daemon/0.log" Jul 10 03:02:05.306769 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:05.306750 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j6wg4_179493af-371a-414e-9602-5de5cae4d7a8/kube-rbac-proxy/0.log" Jul 10 03:02:06.195307 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.195281 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-controller/0.log" Jul 10 03:02:06.213205 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.213183 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/1.log" Jul 10 03:02:06.213356 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.213189 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovn-acl-logging/0.log" Jul 10 03:02:06.236469 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.236448 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/kube-rbac-proxy-node/0.log" Jul 10 03:02:06.261748 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.261728 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/kube-rbac-proxy-ovn-metrics/0.log" Jul 10 03:02:06.283005 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.282975 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/northd/0.log" Jul 10 03:02:06.305196 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.305180 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/nbdb/0.log" Jul 10 03:02:06.328099 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.328066 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/sbdb/0.log" Jul 10 03:02:06.403948 ip-10-0-133-192 kubenswrapper[2560]: I0710 03:02:06.403927 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htnk4_38976ef2-7c34-4df8-9252-337e2e01b774/ovnkube-controller/0.log"