Jul 12 21:52:31.176572 ip-10-0-132-6 systemd[1]: Starting Kubernetes Kubelet... Jul 12 21:52:31.595543 ip-10-0-132-6 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 21:52:31.595543 ip-10-0-132-6 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 21:52:31.595543 ip-10-0-132-6 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 21:52:31.595543 ip-10-0-132-6 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 21:52:31.595543 ip-10-0-132-6 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 21:52:31.596708 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.596605 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 21:52:31.601404 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601364 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.601549 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601498 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.601549 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601515 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.601788 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601646 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601793 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601801 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601805 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601809 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601812 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601815 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601819 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601821 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.601820 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601825 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601828 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601831 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601834 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601837 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601840 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601843 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601845 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601848 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601851 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601854 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601857 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601859 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601862 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601864 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601868 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601871 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601874 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601877 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601880 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.602043 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601883 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601885 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601888 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601891 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601894 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601897 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601899 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601903 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601905 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601908 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601911 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601914 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601916 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601919 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601922 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601925 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601930 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601933 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601935 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601938 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.602558 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601941 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601944 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601947 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601950 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601954 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601957 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601960 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601963 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601966 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601970 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601974 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601977 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601980 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601983 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601985 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601988 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601991 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601994 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.601997 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602000 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.603052 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602005 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602008 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602011 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602014 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602017 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602019 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602022 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602026 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602030 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602033 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602036 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602040 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602043 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602046 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602049 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602052 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602055 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602058 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602061 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602064 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.603572 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602067 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602070 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602072 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602075 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602077 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602080 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602083 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602921 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602933 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602936 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602938 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602941 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602944 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602947 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602949 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602952 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602954 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602957 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602961 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602963 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.604055 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602966 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602969 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602972 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602976 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602979 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602982 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602984 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602987 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602989 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602992 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602995 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.602998 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603001 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603003 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603006 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603008 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603011 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603013 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603017 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603021 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.604577 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603025 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603028 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603031 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603034 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603037 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603040 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603043 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603045 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603048 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603051 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603053 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603056 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603059 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603062 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603065 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603067 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603070 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603073 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603076 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.605078 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603080 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603083 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603086 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603089 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603091 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603094 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603097 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603099 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603101 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603104 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603107 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603109 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603113 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603115 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603118 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603120 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603123 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603125 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603128 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603131 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.605573 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603133 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603137 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603139 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603142 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603145 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603148 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603150 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603153 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603156 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603158 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603161 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603164 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603167 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603172 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603174 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603177 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603180 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603183 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603185 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.606070 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603189 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603193 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603196 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603199 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603202 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603221 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603225 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603228 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603696 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603703 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603706 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603709 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603712 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603716 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603719 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603722 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603724 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603727 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603730 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603733 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.606593 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603736 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603739 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603742 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603745 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603748 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603751 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603754 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603757 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603759 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603762 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603764 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603767 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603770 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603772 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603775 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603778 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603780 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603782 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603785 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.607112 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603789 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603793 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603796 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603799 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603803 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603808 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603812 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603815 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603819 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603822 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603824 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603827 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603830 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603832 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603836 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603838 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603841 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603843 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603847 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603849 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.607595 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603852 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603854 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603857 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603860 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603862 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603864 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603867 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603870 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603873 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603876 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603879 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603881 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603884 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603886 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603889 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603892 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603895 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603899 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603902 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603904 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.608089 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603906 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603910 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603914 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603916 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603919 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603922 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603924 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603929 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603933 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603936 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603938 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603942 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603945 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603947 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603950 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603953 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603955 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603958 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603961 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603963 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.608742 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603966 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603968 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603971 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603973 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603976 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603978 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603980 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.603983 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604065 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604075 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604082 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604087 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604091 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604095 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604099 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604104 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604107 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604109 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604113 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604116 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604119 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604122 2575 flags.go:64] FLAG: --cgroup-root="" Jul 12 21:52:31.609264 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604125 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604128 2575 flags.go:64] FLAG: --client-ca-file="" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604132 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604135 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604139 2575 flags.go:64] FLAG: --cluster-domain="" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604142 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604145 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604149 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604152 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604156 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604159 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604162 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604165 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604169 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604172 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604174 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604177 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604181 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604185 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604189 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604192 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604194 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604197 2575 flags.go:64] FLAG: --enable-server="true" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604200 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604223 2575 flags.go:64] FLAG: --event-burst="100" Jul 12 21:52:31.609831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604227 2575 flags.go:64] FLAG: --event-qps="50" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604230 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604233 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604236 2575 flags.go:64] FLAG: --eviction-hard="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604240 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604243 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604247 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604250 2575 flags.go:64] FLAG: --eviction-soft="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604253 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604256 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604260 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604263 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604266 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604269 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604272 2575 flags.go:64] FLAG: --feature-gates="" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604276 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604280 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604283 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604286 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604290 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604293 2575 flags.go:64] FLAG: --help="false" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604296 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604299 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604302 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 21:52:31.610487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604305 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604309 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604313 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604316 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604319 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604322 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604325 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604328 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604331 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604334 2575 flags.go:64] FLAG: --kube-reserved="" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604337 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604339 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604343 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604345 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604348 2575 flags.go:64] FLAG: --lock-file="" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604351 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604354 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604357 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604363 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604366 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604369 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604372 2575 flags.go:64] FLAG: --logging-format="text" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604375 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604383 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 21:52:31.611108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604386 2575 flags.go:64] FLAG: --manifest-url="" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604389 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604395 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604398 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604402 2575 flags.go:64] FLAG: --max-pods="110" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604405 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604408 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604411 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604414 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604417 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604420 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604422 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604430 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604434 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604437 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604440 2575 flags.go:64] FLAG: --pod-cidr="" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604443 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604449 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604452 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604455 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604458 2575 flags.go:64] FLAG: --port="10250" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604461 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604464 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01e341785e55a388b" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604467 2575 flags.go:64] FLAG: --qos-reserved="" Jul 12 21:52:31.611775 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604471 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604473 2575 flags.go:64] FLAG: --register-node="true" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604476 2575 flags.go:64] FLAG: --register-with-taints="" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604480 2575 flags.go:64] FLAG: --registry-burst="10" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604485 2575 flags.go:64] FLAG: --registry-qps="5" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604488 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604491 2575 flags.go:64] FLAG: --reserved-memory="" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604494 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604504 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604507 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604510 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604513 2575 flags.go:64] FLAG: --runonce="false" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604516 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604519 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604522 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604525 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604528 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604531 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604534 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604537 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604540 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604544 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604547 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604549 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604553 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 21:52:31.612399 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604556 2575 flags.go:64] FLAG: --system-cgroups="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604558 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604564 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604567 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604570 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604574 2575 flags.go:64] FLAG: --tls-min-version="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604577 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604580 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604583 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604586 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604590 2575 flags.go:64] FLAG: --v="2" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604594 2575 flags.go:64] FLAG: --version="false" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604599 2575 flags.go:64] FLAG: --vmodule="" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604603 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.604607 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604740 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604746 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604749 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604752 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604755 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604758 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604760 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604763 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.613024 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604766 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604768 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604772 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604774 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604777 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604780 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604782 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604785 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604789 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604793 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604796 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604798 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604801 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604804 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604807 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604810 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604812 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604815 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604818 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.613653 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604820 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604823 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604826 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604829 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604832 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604839 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604841 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604844 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604847 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604850 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604852 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604855 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604857 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604860 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604863 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604867 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604871 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604874 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604877 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604879 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.614126 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604882 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604885 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604888 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604890 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604893 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604895 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604898 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604901 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604903 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604907 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604911 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604914 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604917 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604920 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604922 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604925 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604929 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604935 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604937 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.614665 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604940 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604943 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604946 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604948 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604951 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604953 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604955 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604958 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604961 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604964 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604966 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604969 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604971 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604974 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604977 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604980 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604983 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604985 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604988 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.615141 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604990 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604993 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604995 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.604998 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605000 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605003 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605006 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605008 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605011 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605013 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605016 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605029 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605032 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.605034 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.605787 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 21:52:31.615640 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.613283 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.613307 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.613328 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.613335 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613397 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613402 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613406 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613409 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613413 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613418 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613422 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613425 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613428 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613432 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613435 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613438 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613442 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613446 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613450 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.616025 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613453 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613455 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613458 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613461 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613464 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613466 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613469 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613472 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613474 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613478 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613481 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613484 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613486 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613490 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613494 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613497 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613500 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613503 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613505 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613508 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.616522 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613512 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613515 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613518 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613520 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613523 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613525 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613528 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613531 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613534 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613538 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613541 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613544 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613547 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613550 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613553 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613556 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613560 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613563 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613566 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.617119 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613568 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613571 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613574 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613577 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613580 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613582 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613585 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613589 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613592 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613594 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613597 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613600 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613602 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613605 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613608 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613611 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613613 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613616 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613619 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613621 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.617842 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613624 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613628 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613630 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613633 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613636 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613638 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613641 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613643 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613646 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613649 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613651 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613654 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613656 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613659 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613662 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613665 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613677 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613680 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613683 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613687 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.618413 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613690 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613693 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613695 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613698 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613701 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.613707 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613837 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613843 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613846 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613849 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613852 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613855 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613858 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613860 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613863 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 21:52:31.618909 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613866 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613869 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613872 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613875 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613878 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613880 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613883 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613886 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613888 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613893 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613897 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613900 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613903 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613906 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613910 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613913 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613916 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613918 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613923 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 21:52:31.619315 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613925 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613928 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613931 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613933 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613936 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613939 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613941 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613945 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613948 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613951 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613953 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613956 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613959 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613962 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613964 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613967 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613969 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613972 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613975 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613977 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 21:52:31.619780 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613980 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613982 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613985 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613987 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613990 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613992 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613994 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.613998 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614001 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614004 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614006 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614009 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614011 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614014 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614016 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614019 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614022 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614024 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614027 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614029 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 21:52:31.620482 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614032 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614034 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614037 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614040 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614043 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614045 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614048 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614051 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614053 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614056 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614059 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614061 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614064 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614067 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614069 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614072 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614074 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614077 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614080 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614083 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 21:52:31.621023 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614086 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614088 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614092 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614096 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614098 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614101 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614104 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614107 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614109 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614112 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:31.614115 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.614120 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.614296 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.616667 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 21:52:31.621537 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.618087 2575 server.go:1019] "Starting client certificate rotation" Jul 12 21:52:31.621899 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.618219 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 21:52:31.621899 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.618263 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 21:52:31.621899 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.620978 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 21:52:31.623021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.622998 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 21:52:31.635485 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.635454 2575 log.go:25] "Validated CRI v1 runtime API" Jul 12 21:52:31.637877 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.637852 2575 log.go:25] "Validated CRI v1 image API" Jul 12 21:52:31.639461 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.639436 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 21:52:31.642570 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.642528 2575 fs.go:135] Filesystem UUIDs: map[6212d23d-9a55-4f34-b4ed-dd8e0e0c6a0c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fb0181ca-7691-4fb1-818a-86cea9af1479:/dev/nvme0n1p3] Jul 12 21:52:31.642570 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.642561 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 21:52:31.646377 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.646346 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 21:52:31.648968 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.648811 2575 manager.go:216] Machine: {Timestamp:2026-07-12 21:52:31.64741922 +0000 UTC m=+0.344906790 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095698 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26b8e97ada869989c7321e334a6e56 SystemUUID:ec26b8e9-7ada-8699-89c7-321e334a6e56 BootID:38403678-693a-43db-9c8b-180f0541d1f6 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a3:87:7b:9d:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a3:87:7b:9d:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:a9:f1:32:74:14 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 21:52:31.648968 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.648951 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 21:52:31.649183 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.649093 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 21:52:31.652182 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.652134 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 21:52:31.652378 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.652186 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 21:52:31.652913 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.652900 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 21:52:31.652945 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.652916 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 21:52:31.652945 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.652925 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 21:52:31.653666 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.653651 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 21:52:31.655066 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.655051 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 21:52:31.658274 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658249 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 12 21:52:31.658430 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658419 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 21:52:31.658473 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658454 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 12 21:52:31.658473 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658466 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 21:52:31.658542 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658482 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 21:52:31.658542 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658497 2575 kubelet.go:398] "Adding apiserver pod source" Jul 12 21:52:31.658542 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658507 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 21:52:31.658943 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658930 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 21:52:31.658982 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.658952 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 21:52:31.661136 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.661113 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 21:52:31.663222 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.663186 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 21:52:31.663334 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.663237 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 21:52:31.664006 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.663992 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 21:52:31.664046 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664013 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 21:52:31.664046 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664019 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 21:52:31.664046 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664030 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 21:52:31.664046 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664039 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 21:52:31.664046 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664047 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664056 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664061 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664069 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664077 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664097 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 21:52:31.664177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664104 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 21:52:31.664833 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664819 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 21:52:31.664895 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.664835 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 21:52:31.667415 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.667379 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 21:52:31.667909 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.667854 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 21:52:31.667909 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.667850 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 21:52:31.668477 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.668464 2575 server.go:1268] "Started kubelet" Jul 12 21:52:31.668562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.668530 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 21:52:31.668637 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.668599 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 21:52:31.668673 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.668659 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 21:52:31.669682 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.669655 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rxjk8" Jul 12 21:52:31.669970 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.669949 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 21:52:31.671005 ip-10-0-132-6 systemd[1]: Started Kubernetes Kubelet. Jul 12 21:52:31.674440 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.674379 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 12 21:52:31.675187 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.675165 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 21:52:31.675976 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.673463 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-6.ec2.internal.18c1a95eb672ef85 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-6.ec2.internal,UID:ip-10-0-132-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-6.ec2.internal,},FirstTimestamp:2026-07-12 21:52:31.668432773 +0000 UTC m=+0.365920402,LastTimestamp:2026-07-12 21:52:31.668432773 +0000 UTC m=+0.365920402,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-6.ec2.internal,}" Jul 12 21:52:31.677230 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.677198 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rxjk8" Jul 12 21:52:31.678115 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.678097 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 21:52:31.678161 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.678100 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 21:52:31.678416 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.678393 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:31.678907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.678858 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 21:52:31.678907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.678867 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 21:52:31.678907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.678877 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 21:52:31.679090 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679043 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 12 21:52:31.679090 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679052 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 12 21:52:31.679313 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679296 2575 factory.go:55] Registering systemd factory Jul 12 21:52:31.679313 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679313 2575 factory.go:223] Registration of the systemd container factory successfully Jul 12 21:52:31.679603 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679575 2575 factory.go:153] Registering CRI-O factory Jul 12 21:52:31.679603 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679593 2575 factory.go:223] Registration of the crio container factory successfully Jul 12 21:52:31.679736 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679656 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 21:52:31.679736 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679686 2575 factory.go:103] Registering Raw factory Jul 12 21:52:31.679736 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.679701 2575 manager.go:1196] Started watching for new ooms in manager Jul 12 21:52:31.680640 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.680616 2575 manager.go:349] Starting recovery of all containers Jul 12 21:52:31.680730 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.680681 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 21:52:31.680831 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.680807 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 21:52:31.692293 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.692040 2575 manager.go:354] Recovery completed Jul 12 21:52:31.698120 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.698095 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.700186 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700167 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.700278 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700199 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.700278 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700226 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.700790 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700770 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 21:52:31.700790 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700784 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 21:52:31.700940 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700807 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 12 21:52:31.700940 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.700830 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.701674 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.701658 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.701721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.701684 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.701721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.701695 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.702278 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.702252 2575 policy_none.go:49] "None policy: Start" Jul 12 21:52:31.702278 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.702281 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 21:52:31.702371 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.702293 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 21:52:31.703596 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.703583 2575 policy_none.go:47] "Start" Jul 12 21:52:31.740069 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.740044 2575 manager.go:337] "Starting Device Plugin manager" Jul 12 21:52:31.740218 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.740111 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 21:52:31.740218 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.740127 2575 server.go:87] "Starting device plugin registration server" Jul 12 21:52:31.740458 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.740444 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 21:52:31.740525 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.740459 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 21:52:31.741320 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.741292 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 21:52:31.741320 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.741287 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 21:52:31.741554 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.741365 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:31.741554 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.741425 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 21:52:31.741554 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.741435 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 21:52:31.779725 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.779690 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 21:52:31.782007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.781018 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 21:52:31.782007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.781057 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 21:52:31.782007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.781084 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 21:52:31.782007 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.781122 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 21:52:31.783957 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.783852 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:31.841606 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.841548 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.842892 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.842872 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.843007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.842906 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.843007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.842922 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.843007 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.842953 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.851306 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.851241 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.851306 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.851278 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-6.ec2.internal\": node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:31.868494 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.868467 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:31.882572 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.881839 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal"] Jul 12 21:52:31.882654 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.882620 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.883671 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.883653 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.883763 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.883687 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.883763 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.883700 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.885162 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.885149 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.885326 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.885310 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.885365 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.885347 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.886016 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886000 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.886103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886006 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.886103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886033 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.886103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886056 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.886103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886060 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.886103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.886072 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.887786 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.887769 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.887861 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.887801 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 21:52:31.888737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.888719 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientMemory" Jul 12 21:52:31.888825 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.888752 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 21:52:31.888825 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:31.888765 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeHasSufficientPID" Jul 12 21:52:31.911382 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.911358 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-6.ec2.internal\" not found" node="ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.916430 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.916406 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-6.ec2.internal\" not found" node="ip-10-0-132-6.ec2.internal" Jul 12 21:52:31.968570 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:31.968529 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.069649 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.069612 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.079996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.079961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.080109 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.080000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.080109 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.080022 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bbbba9f59567f90d3b65dc971a3dd1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"62bbbba9f59567f90d3b65dc971a3dd1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.170432 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.170359 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.180762 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180737 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bbbba9f59567f90d3b65dc971a3dd1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"62bbbba9f59567f90d3b65dc971a3dd1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.180893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.180893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180787 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.180893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.180893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180862 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62bbbba9f59567f90d3b65dc971a3dd1-config\") pod \"kube-apiserver-proxy-ip-10-0-132-6.ec2.internal\" (UID: \"62bbbba9f59567f90d3b65dc971a3dd1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.180893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.180876 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd31a767463220c11745fd3417d6720e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal\" (UID: \"bd31a767463220c11745fd3417d6720e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.213935 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.213891 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.219288 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.219263 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.270528 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.270484 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.371168 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.371131 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.471769 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.471680 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.561683 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.561648 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:32.572282 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.572240 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.618776 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.618745 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 21:52:32.619347 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.618900 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 21:52:32.619347 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.618923 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 21:52:32.672843 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.672808 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.681442 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.681395 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 21:47:32 +0000 UTC" deadline="2028-04-01 17:27:54.30431286 +0000 UTC" Jul 12 21:52:32.681442 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.681433 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15091h35m21.622884757s" Jul 12 21:52:32.753328 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:32.753287 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd31a767463220c11745fd3417d6720e.slice/crio-fd807f64911908b2be758fcbec8c0f59d16748298c3ba633904edbc544692eff WatchSource:0}: Error finding container fd807f64911908b2be758fcbec8c0f59d16748298c3ba633904edbc544692eff: Status 404 returned error can't find the container with id fd807f64911908b2be758fcbec8c0f59d16748298c3ba633904edbc544692eff Jul 12 21:52:32.753771 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:32.753746 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bbbba9f59567f90d3b65dc971a3dd1.slice/crio-84609ab26cfcf968ea711c543a72546590b62033ffdd4d9f2e940a2b5661c05f WatchSource:0}: Error finding container 84609ab26cfcf968ea711c543a72546590b62033ffdd4d9f2e940a2b5661c05f: Status 404 returned error can't find the container with id 84609ab26cfcf968ea711c543a72546590b62033ffdd4d9f2e940a2b5661c05f Jul 12 21:52:32.760502 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.760481 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 21:52:32.772923 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.772892 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.785783 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.785723 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"bd31a767463220c11745fd3417d6720e","Type":"ContainerStarted","Data":"fd807f64911908b2be758fcbec8c0f59d16748298c3ba633904edbc544692eff"} Jul 12 21:52:32.786709 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.786687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" event={"ID":"62bbbba9f59567f90d3b65dc971a3dd1","Type":"ContainerStarted","Data":"84609ab26cfcf968ea711c543a72546590b62033ffdd4d9f2e940a2b5661c05f"} Jul 12 21:52:32.800387 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.800356 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:32.876704 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:32.876661 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-6.ec2.internal\" not found" Jul 12 21:52:32.964821 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.964788 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:32.978626 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.978598 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" Jul 12 21:52:32.991975 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.991942 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 21:52:32.993323 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:32.993257 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" Jul 12 21:52:33.004569 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.004535 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 21:52:33.659728 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.659686 2575 apiserver.go:52] "Watching apiserver" Jul 12 21:52:33.668473 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.668436 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 21:52:33.669163 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.669130 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-6pcwz","openshift-ovn-kubernetes/ovnkube-node-hzvt7","kube-system/global-pull-secret-syncer-c7l4v","kube-system/konnectivity-agent-nr65r","kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87","openshift-cluster-node-tuning-operator/tuned-54wbd","openshift-dns/node-resolver-rsbww","openshift-image-registry/node-ca-wktg5","openshift-multus/multus-7bnnm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal","openshift-multus/multus-additional-cni-plugins-bzrhw","openshift-multus/network-metrics-daemon-mtcjn","openshift-network-diagnostics/network-check-target-st2cg"] Jul 12 21:52:33.670934 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.670903 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.672243 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.672191 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.673618 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.673593 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.673744 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.673693 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:33.674269 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.674092 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.674269 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.674119 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mkmcn\"" Jul 12 21:52:33.675124 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.674628 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.675245 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.675220 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.675573 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.675552 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.676031 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.676009 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g2l6g\"" Jul 12 21:52:33.676336 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.676316 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.679997 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.679960 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 21:52:33.681067 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.681040 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 21:52:33.681569 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.681547 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 21:52:33.681746 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.681547 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rdttt\"" Jul 12 21:52:33.682979 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.682956 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.684996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.684973 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.686963 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.686710 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.686963 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.686749 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.687482 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687462 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 21:52:33.687583 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687517 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.687705 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687687 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.687761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687691 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mpl42\"" Jul 12 21:52:33.687761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687741 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 21:52:33.687860 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.687796 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.688266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688221 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-kubernetes\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-lib-modules\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-host\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688308 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-tuned\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpdjb\" (UniqueName: \"kubernetes.io/projected/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-kube-api-access-fpdjb\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-kubelet-config\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-dbus\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.688428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688409 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd8d6b7a-3815-4786-91f8-d11babe079dd-host-slash\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-sys\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.688431 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688454 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-tmp\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688494 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fd652c-b3e0-4244-8b32-c80081d686bc-agent-certs\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688519 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cd8d6b7a-3815-4786-91f8-d11babe079dd-iptables-alerter-script\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-modprobe-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-run\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-var-lib-kubelet\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1545efb1-e4a2-4ab6-8001-4137a9a081fe-hosts-file\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fd652c-b3e0-4244-8b32-c80081d686bc-konnectivity-ca\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688709 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jbfc\" (UniqueName: \"kubernetes.io/projected/cd8d6b7a-3815-4786-91f8-d11babe079dd-kube-api-access-8jbfc\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688733 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysconfig\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688757 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688802 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k86zj\"" Jul 12 21:52:33.688841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688807 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-conf\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.689724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-systemd\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.689724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688899 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1545efb1-e4a2-4ab6-8001-4137a9a081fe-tmp-dir\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.689724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.688928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr7h8\" (UniqueName: \"kubernetes.io/projected/1545efb1-e4a2-4ab6-8001-4137a9a081fe-kube-api-access-tr7h8\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.689724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.689588 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.689925 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.689819 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2bz6m\"" Jul 12 21:52:33.690052 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.690017 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 21:52:33.690675 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.690506 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.690675 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.690640 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.692856 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.692567 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.694006 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.693988 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:33.694096 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.694055 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:33.694449 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694427 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 21:52:33.694554 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694453 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.694554 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694462 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 21:52:33.694667 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694587 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 21:52:33.694721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694708 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 21:52:33.694935 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.694917 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.695086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695059 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sr22x\"" Jul 12 21:52:33.695198 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695114 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 21:52:33.695337 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695317 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 21:52:33.695406 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695338 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 21:52:33.695838 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695819 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.695925 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.695878 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 21:52:33.696672 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.696637 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 21:52:33.696786 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.696686 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lqkb9\"" Jul 12 21:52:33.698601 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.698581 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 21:52:33.699386 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.698944 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 21:52:33.699386 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.699173 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6s24r\"" Jul 12 21:52:33.721116 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.721067 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6xq6" Jul 12 21:52:33.732144 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.732106 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6xq6" Jul 12 21:52:33.780590 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.780539 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 21:52:33.789284 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-tuned\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789297 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-kubelet-config\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789341 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-dbus\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-os-release\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789407 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-etc-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789431 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-os-release\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-kubelet-config\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.789480 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789471 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-dbus\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789484 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-kubelet\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-etc-kubernetes\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789533 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789613 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789656 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-env-overrides\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789660 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-modprobe-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.789811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789787 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-run\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-var-lib-kubelet\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789871 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789881 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-modprobe-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789904 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-log-socket\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789944 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-run\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789947 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-config\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.789998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4mx6\" (UniqueName: \"kubernetes.io/projected/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-kube-api-access-m4mx6\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790003 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-var-lib-kubelet\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-conf\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790054 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmcpb\" (UniqueName: \"kubernetes.io/projected/6d64f326-fae3-48ef-834f-10d3f9080aaa-kube-api-access-tmcpb\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790108 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-systemd-units\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790136 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-system-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-cni-binary-copy\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.790216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-multus\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-device-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790204 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-conf\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-ovn\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790293 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-netd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790362 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-netns\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790388 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-hostroot\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd8d6b7a-3815-4786-91f8-d11babe079dd-host-slash\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-sys\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-tmp\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cd8d6b7a-3815-4786-91f8-d11babe079dd-host-slash\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790562 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-kubelet\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790565 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-sys\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-node-log\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790621 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-bin\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790646 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-bin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cd8d6b7a-3815-4786-91f8-d11babe079dd-iptables-alerter-script\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1545efb1-e4a2-4ab6-8001-4137a9a081fe-hosts-file\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790888 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-netns\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790898 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1545efb1-e4a2-4ab6-8001-4137a9a081fe-hosts-file\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790924 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jbfc\" (UniqueName: \"kubernetes.io/projected/cd8d6b7a-3815-4786-91f8-d11babe079dd-kube-api-access-8jbfc\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790953 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysconfig\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.790979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-systemd\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr7h8\" (UniqueName: \"kubernetes.io/projected/1545efb1-e4a2-4ab6-8001-4137a9a081fe-kube-api-access-tr7h8\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysconfig\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-system-cni-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpdjb\" (UniqueName: \"kubernetes.io/projected/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-kube-api-access-fpdjb\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791169 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-sysctl-d\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fd652c-b3e0-4244-8b32-c80081d686bc-agent-certs\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.791953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-host\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8rfw\" (UniqueName: \"kubernetes.io/projected/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-kube-api-access-j8rfw\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791095 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-systemd\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791320 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-script-lib\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791320 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cd8d6b7a-3815-4786-91f8-d11babe079dd-iptables-alerter-script\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjpgm\" (UniqueName: \"kubernetes.io/projected/2f9b93ec-4582-4341-be8a-e777ff69340c-kube-api-access-gjpgm\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791547 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-systemd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791657 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fd652c-b3e0-4244-8b32-c80081d686bc-konnectivity-ca\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791682 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-conf-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-daemon-config\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791735 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791759 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791787 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb2dw\" (UniqueName: \"kubernetes.io/projected/4841a61b-e23a-4fab-ab80-6705746aaf9a-kube-api-access-qb2dw\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.792815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791814 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-kubernetes\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791839 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-lib-modules\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.791848 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791862 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-host\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-socket-dir-parent\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791911 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-multus-certs\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.791928 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:34.291910932 +0000 UTC m=+2.989398491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.791991 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-var-lib-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792015 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovn-node-metrics-cert\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792039 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-cnibin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5hd\" (UniqueName: \"kubernetes.io/projected/7d4a260a-963c-4a9d-938b-29ec4d903536-kube-api-access-lq5hd\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-k8s-cni-cncf-io\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792153 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-kubernetes\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1545efb1-e4a2-4ab6-8001-4137a9a081fe-tmp-dir\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792216 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-serviceca\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792230 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-host\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.793432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792260 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-cnibin\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-slash\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792354 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-lib-modules\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792260 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48fd652c-b3e0-4244-8b32-c80081d686bc-konnectivity-ca\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.792568 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1545efb1-e4a2-4ab6-8001-4137a9a081fe-tmp-dir\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.794024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-tmp\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.794259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.794138 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-etc-tuned\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.795408 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.795381 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48fd652c-b3e0-4244-8b32-c80081d686bc-agent-certs\") pod \"konnectivity-agent-nr65r\" (UID: \"48fd652c-b3e0-4244-8b32-c80081d686bc\") " pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:33.802729 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.802685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr7h8\" (UniqueName: \"kubernetes.io/projected/1545efb1-e4a2-4ab6-8001-4137a9a081fe-kube-api-access-tr7h8\") pod \"node-resolver-rsbww\" (UID: \"1545efb1-e4a2-4ab6-8001-4137a9a081fe\") " pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:33.802893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.802807 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpdjb\" (UniqueName: \"kubernetes.io/projected/ea945cc8-1abb-425d-8230-3b9e1b5c8a26-kube-api-access-fpdjb\") pod \"tuned-54wbd\" (UID: \"ea945cc8-1abb-425d-8230-3b9e1b5c8a26\") " pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.803018 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.802994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jbfc\" (UniqueName: \"kubernetes.io/projected/cd8d6b7a-3815-4786-91f8-d11babe079dd-kube-api-access-8jbfc\") pod \"iptables-alerter-6pcwz\" (UID: \"cd8d6b7a-3815-4786-91f8-d11babe079dd\") " pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:33.893532 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893487 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.893532 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893536 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-log-socket\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-config\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893579 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mx6\" (UniqueName: \"kubernetes.io/projected/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-kube-api-access-m4mx6\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893607 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893633 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-log-socket\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893651 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmcpb\" (UniqueName: \"kubernetes.io/projected/6d64f326-fae3-48ef-834f-10d3f9080aaa-kube-api-access-tmcpb\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-systemd-units\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-system-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-cni-binary-copy\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-multus\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.893812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893801 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-device-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-ovn\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893847 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-netd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893869 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893871 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-systemd-units\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-netns\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893933 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-netns\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-hostroot\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.893990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-system-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-kubelet\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-node-log\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894066 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-hostroot\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-bin\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894133 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-bin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894169 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-ovn\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894208 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-netns\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894223 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.894452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894229 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894254 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-node-log\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894170 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-netns\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894302 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-bin\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894307 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-cni-netd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894307 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-system-cni-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894136 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-cni-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894307 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-config\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894358 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-multus\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894375 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-cni-bin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894373 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-system-cni-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894422 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-kubelet\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894427 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-device-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894534 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.895472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-host\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8rfw\" (UniqueName: \"kubernetes.io/projected/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-kube-api-access-j8rfw\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894577 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-host\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894593 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-script-lib\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894626 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894646 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjpgm\" (UniqueName: \"kubernetes.io/projected/2f9b93ec-4582-4341-be8a-e777ff69340c-kube-api-access-gjpgm\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894708 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-systemd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894718 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-socket-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894735 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-conf-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894752 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894769 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-conf-dir\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-daemon-config\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894788 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-cni-binary-copy\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894815 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-run-systemd\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.896432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb2dw\" (UniqueName: \"kubernetes.io/projected/4841a61b-e23a-4fab-ab80-6705746aaf9a-kube-api-access-qb2dw\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-socket-dir-parent\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-multus-certs\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-multus-certs\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.894968 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895011 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-socket-dir-parent\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895136 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-var-lib-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovn-node-metrics-cert\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895191 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovnkube-script-lib\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895195 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-cnibin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895210 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d4a260a-963c-4a9d-938b-29ec4d903536-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895226 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-var-lib-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895195 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-registration-dir\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895266 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-cnibin\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895277 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2f9b93ec-4582-4341-be8a-e777ff69340c-multus-daemon-config\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.897401 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895311 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5hd\" (UniqueName: \"kubernetes.io/projected/7d4a260a-963c-4a9d-938b-29ec4d903536-kube-api-access-lq5hd\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-k8s-cni-cncf-io\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-serviceca\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895435 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-run-k8s-cni-cncf-io\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-cnibin\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-slash\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-os-release\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-etc-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895583 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-cnibin\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895583 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-slash\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-os-release\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-kubelet\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895648 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d4a260a-963c-4a9d-938b-29ec4d903536-os-release\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895657 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-os-release\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-etc-kubernetes\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-host-var-lib-kubelet\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895697 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-etc-openvswitch\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898209 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895735 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2f9b93ec-4582-4341-be8a-e777ff69340c-etc-kubernetes\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-env-overrides\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895791 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d64f326-fae3-48ef-834f-10d3f9080aaa-sys-fs\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.895806 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-serviceca\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.895813 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.895890 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:52:34.395868081 +0000 UTC m=+3.093355645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.896188 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-env-overrides\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.898806 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.898409 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-ovn-node-metrics-cert\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.907413 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.907169 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:33.907413 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.907214 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:33.907413 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.907233 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:33.907971 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:33.907943 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:34.407893672 +0000 UTC m=+3.105381283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:33.910406 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.910310 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5hd\" (UniqueName: \"kubernetes.io/projected/7d4a260a-963c-4a9d-938b-29ec4d903536-kube-api-access-lq5hd\") pod \"multus-additional-cni-plugins-bzrhw\" (UID: \"7d4a260a-963c-4a9d-938b-29ec4d903536\") " pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:33.912617 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.912539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4mx6\" (UniqueName: \"kubernetes.io/projected/1c58b8f6-f0c0-4142-86d0-de5049ec66b5-kube-api-access-m4mx6\") pod \"ovnkube-node-hzvt7\" (UID: \"1c58b8f6-f0c0-4142-86d0-de5049ec66b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:33.913737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.913472 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmcpb\" (UniqueName: \"kubernetes.io/projected/6d64f326-fae3-48ef-834f-10d3f9080aaa-kube-api-access-tmcpb\") pod \"aws-ebs-csi-driver-node-cnn87\" (UID: \"6d64f326-fae3-48ef-834f-10d3f9080aaa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:33.913737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.913685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8rfw\" (UniqueName: \"kubernetes.io/projected/4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9-kube-api-access-j8rfw\") pod \"node-ca-wktg5\" (UID: \"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9\") " pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:33.913737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.913716 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjpgm\" (UniqueName: \"kubernetes.io/projected/2f9b93ec-4582-4341-be8a-e777ff69340c-kube-api-access-gjpgm\") pod \"multus-7bnnm\" (UID: \"2f9b93ec-4582-4341-be8a-e777ff69340c\") " pod="openshift-multus/multus-7bnnm" Jul 12 21:52:33.914953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.914910 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb2dw\" (UniqueName: \"kubernetes.io/projected/4841a61b-e23a-4fab-ab80-6705746aaf9a-kube-api-access-qb2dw\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:33.986752 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.986707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-54wbd" Jul 12 21:52:33.997371 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:33.997327 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rsbww" Jul 12 21:52:34.009888 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.009851 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:34.016151 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.016109 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6pcwz" Jul 12 21:52:34.024692 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.024644 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" Jul 12 21:52:34.033413 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.033376 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wktg5" Jul 12 21:52:34.044633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.044592 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:52:34.051953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.051923 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7bnnm" Jul 12 21:52:34.057564 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.057524 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:34.061970 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.061939 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" Jul 12 21:52:34.299719 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.299609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:34.299900 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.299804 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:34.299900 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.299882 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:35.299861834 +0000 UTC m=+3.997349396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:34.400883 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.400845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:34.401079 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.401015 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:34.401133 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.401091 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:52:35.401073077 +0000 UTC m=+4.098560638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:34.501704 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.501654 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:34.501909 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.501812 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:34.501909 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.501832 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:34.501909 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.501844 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:34.502116 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.501914 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:35.501892328 +0000 UTC m=+4.199379889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:34.734820 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.734684 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 21:47:34 +0000 UTC" deadline="2028-02-06 05:38:39.988248941 +0000 UTC" Jul 12 21:52:34.734820 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.734760 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13759h46m5.253503265s" Jul 12 21:52:34.777647 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.777479 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d64f326_fae3_48ef_834f_10d3f9080aaa.slice/crio-bd3244bc41da1b19e46eae3e62908895934f8088afb62f2730f2e2333a0785a3 WatchSource:0}: Error finding container bd3244bc41da1b19e46eae3e62908895934f8088afb62f2730f2e2333a0785a3: Status 404 returned error can't find the container with id bd3244bc41da1b19e46eae3e62908895934f8088afb62f2730f2e2333a0785a3 Jul 12 21:52:34.781652 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.781558 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:34.781776 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:34.781694 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:34.784775 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.784653 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e55c5f9_fffd_4e4d_ad8f_75ba9e975ef9.slice/crio-aed0a8f6aa524126cbc9f0b6ecbe95de35c595cd0b53ace5e3c6cc4cca72cfa7 WatchSource:0}: Error finding container aed0a8f6aa524126cbc9f0b6ecbe95de35c595cd0b53ace5e3c6cc4cca72cfa7: Status 404 returned error can't find the container with id aed0a8f6aa524126cbc9f0b6ecbe95de35c595cd0b53ace5e3c6cc4cca72cfa7 Jul 12 21:52:34.790849 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.790816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wktg5" event={"ID":"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9","Type":"ContainerStarted","Data":"aed0a8f6aa524126cbc9f0b6ecbe95de35c595cd0b53ace5e3c6cc4cca72cfa7"} Jul 12 21:52:34.792063 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.792037 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" event={"ID":"6d64f326-fae3-48ef-834f-10d3f9080aaa","Type":"ContainerStarted","Data":"bd3244bc41da1b19e46eae3e62908895934f8088afb62f2730f2e2333a0785a3"} Jul 12 21:52:34.799428 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.799393 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea945cc8_1abb_425d_8230_3b9e1b5c8a26.slice/crio-11273310d4b2850c6573dca3fbb0676d6885604af074a9fdb7de8dd74df985df WatchSource:0}: Error finding container 11273310d4b2850c6573dca3fbb0676d6885604af074a9fdb7de8dd74df985df: Status 404 returned error can't find the container with id 11273310d4b2850c6573dca3fbb0676d6885604af074a9fdb7de8dd74df985df Jul 12 21:52:34.800173 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.800137 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fd652c_b3e0_4244_8b32_c80081d686bc.slice/crio-b5698435482e57ec7511e1ccba434b0de39d0a7220a27f455574948690406a4d WatchSource:0}: Error finding container b5698435482e57ec7511e1ccba434b0de39d0a7220a27f455574948690406a4d: Status 404 returned error can't find the container with id b5698435482e57ec7511e1ccba434b0de39d0a7220a27f455574948690406a4d Jul 12 21:52:34.801962 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.801896 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8d6b7a_3815_4786_91f8_d11babe079dd.slice/crio-99aff57ada8a7b891cdbbbb5ba91c06127e9420aa4a4ab9cc1b5621a90e2746a WatchSource:0}: Error finding container 99aff57ada8a7b891cdbbbb5ba91c06127e9420aa4a4ab9cc1b5621a90e2746a: Status 404 returned error can't find the container with id 99aff57ada8a7b891cdbbbb5ba91c06127e9420aa4a4ab9cc1b5621a90e2746a Jul 12 21:52:34.803750 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.803557 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1545efb1_e4a2_4ab6_8001_4137a9a081fe.slice/crio-47b95781bd28812f637cdceddecc52be2ae1ff3f81aa6c35e392347f1ef1512d WatchSource:0}: Error finding container 47b95781bd28812f637cdceddecc52be2ae1ff3f81aa6c35e392347f1ef1512d: Status 404 returned error can't find the container with id 47b95781bd28812f637cdceddecc52be2ae1ff3f81aa6c35e392347f1ef1512d Jul 12 21:52:34.805170 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.805137 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9b93ec_4582_4341_be8a_e777ff69340c.slice/crio-867729dca899228e872aee5ab6276598a00144dc5d0c39a77d15797ac5529967 WatchSource:0}: Error finding container 867729dca899228e872aee5ab6276598a00144dc5d0c39a77d15797ac5529967: Status 404 returned error can't find the container with id 867729dca899228e872aee5ab6276598a00144dc5d0c39a77d15797ac5529967 Jul 12 21:52:34.806763 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.806740 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c58b8f6_f0c0_4142_86d0_de5049ec66b5.slice/crio-994c08c990e8f0e99cda0343bfb900734a4dff29cc641a984d0508992d6663c2 WatchSource:0}: Error finding container 994c08c990e8f0e99cda0343bfb900734a4dff29cc641a984d0508992d6663c2: Status 404 returned error can't find the container with id 994c08c990e8f0e99cda0343bfb900734a4dff29cc641a984d0508992d6663c2 Jul 12 21:52:34.808005 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:52:34.807982 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4a260a_963c_4a9d_938b_29ec4d903536.slice/crio-35b1bb5d76c51fcf6ef93e8c356005ee4bd4aea4c63c638b70b68ebd9de1831f WatchSource:0}: Error finding container 35b1bb5d76c51fcf6ef93e8c356005ee4bd4aea4c63c638b70b68ebd9de1831f: Status 404 returned error can't find the container with id 35b1bb5d76c51fcf6ef93e8c356005ee4bd4aea4c63c638b70b68ebd9de1831f Jul 12 21:52:34.962993 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:34.962737 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 21:52:35.309413 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.309308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:35.309593 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.309483 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:35.309593 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.309558 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:37.309534748 +0000 UTC m=+6.007022311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:35.409854 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.409787 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:35.410073 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.409958 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:35.410073 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.410040 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:52:37.410018638 +0000 UTC m=+6.107506202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:35.510222 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.510185 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:35.510709 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.510682 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:35.510812 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.510724 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:35.510812 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.510741 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:35.510922 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.510837 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:37.510814008 +0000 UTC m=+6.208301572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:35.735570 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.735430 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 21:47:34 +0000 UTC" deadline="2027-12-15 18:04:06.952322299 +0000 UTC" Jul 12 21:52:35.735570 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.735501 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12500h11m31.216826514s" Jul 12 21:52:35.782986 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.782429 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:35.782986 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.782576 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:35.783661 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.783410 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:35.783661 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:35.783535 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:35.806158 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.806109 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nr65r" event={"ID":"48fd652c-b3e0-4244-8b32-c80081d686bc","Type":"ContainerStarted","Data":"b5698435482e57ec7511e1ccba434b0de39d0a7220a27f455574948690406a4d"} Jul 12 21:52:35.852197 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.852149 2575 generic.go:356] "Generic (PLEG): container finished" podID="bd31a767463220c11745fd3417d6720e" containerID="02d0f912f5b8c976323337fbb5978fb639af02630bc0a1dd762c4e87ce4561ac" exitCode=0 Jul 12 21:52:35.852503 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.852317 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"bd31a767463220c11745fd3417d6720e","Type":"ContainerDied","Data":"02d0f912f5b8c976323337fbb5978fb639af02630bc0a1dd762c4e87ce4561ac"} Jul 12 21:52:35.869116 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.869068 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" event={"ID":"62bbbba9f59567f90d3b65dc971a3dd1","Type":"ContainerStarted","Data":"deab1ba9780337b05c0f5e878f7c52eaa4a452df8ca124824ab7736cbdcd2038"} Jul 12 21:52:35.893173 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.893026 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerStarted","Data":"35b1bb5d76c51fcf6ef93e8c356005ee4bd4aea4c63c638b70b68ebd9de1831f"} Jul 12 21:52:35.901998 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.901821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"994c08c990e8f0e99cda0343bfb900734a4dff29cc641a984d0508992d6663c2"} Jul 12 21:52:35.920591 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.920509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7bnnm" event={"ID":"2f9b93ec-4582-4341-be8a-e777ff69340c","Type":"ContainerStarted","Data":"867729dca899228e872aee5ab6276598a00144dc5d0c39a77d15797ac5529967"} Jul 12 21:52:35.927746 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.927677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6pcwz" event={"ID":"cd8d6b7a-3815-4786-91f8-d11babe079dd","Type":"ContainerStarted","Data":"99aff57ada8a7b891cdbbbb5ba91c06127e9420aa4a4ab9cc1b5621a90e2746a"} Jul 12 21:52:35.936749 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.936706 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-54wbd" event={"ID":"ea945cc8-1abb-425d-8230-3b9e1b5c8a26","Type":"ContainerStarted","Data":"11273310d4b2850c6573dca3fbb0676d6885604af074a9fdb7de8dd74df985df"} Jul 12 21:52:35.961114 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:35.961066 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rsbww" event={"ID":"1545efb1-e4a2-4ab6-8001-4137a9a081fe","Type":"ContainerStarted","Data":"47b95781bd28812f637cdceddecc52be2ae1ff3f81aa6c35e392347f1ef1512d"} Jul 12 21:52:36.782174 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:36.781590 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:36.782174 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:36.781755 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:36.974609 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:36.974522 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" event={"ID":"bd31a767463220c11745fd3417d6720e","Type":"ContainerStarted","Data":"b71cde6d7cdfc1ec478806a376addd577ee0afdcbbafd6aa927ea49b65f091ba"} Jul 12 21:52:36.995849 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:36.995767 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-6.ec2.internal" podStartSLOduration=3.995740875 podStartE2EDuration="3.995740875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:35.888903283 +0000 UTC m=+4.586390871" watchObservedRunningTime="2026-07-12 21:52:36.995740875 +0000 UTC m=+5.693228460" Jul 12 21:52:37.326971 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:37.326916 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:37.327334 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.327158 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:37.327334 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.327230 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:41.327207945 +0000 UTC m=+10.024695507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:37.428908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:37.428173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:37.428908 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.428419 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:37.428908 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.428506 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:52:41.428483536 +0000 UTC m=+10.125971120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:37.529544 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:37.528827 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:37.529544 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.529031 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:37.529544 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.529057 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:37.529544 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.529071 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:37.529544 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.529146 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:41.52912493 +0000 UTC m=+10.226612509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:37.784882 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:37.784045 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:37.784882 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.784192 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:37.784882 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:37.784647 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:37.784882 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:37.784761 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:38.782336 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:38.782294 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:38.782540 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:38.782456 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:39.783124 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:39.783084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:39.783655 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:39.783254 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:39.783655 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:39.783394 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:39.783655 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:39.783251 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:40.781392 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:40.781352 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:40.781619 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:40.781541 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:41.362579 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:41.362533 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:41.363068 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.362733 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:41.363068 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.362815 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:49.362791495 +0000 UTC m=+18.060279059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:41.463693 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:41.463647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:41.463906 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.463858 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:41.463972 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.463934 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:52:49.463909407 +0000 UTC m=+18.161396972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:41.564440 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:41.564390 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:41.564639 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.564596 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:41.564639 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.564627 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:41.564767 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.564644 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:41.564832 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.564761 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:52:49.564733476 +0000 UTC m=+18.262221043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:41.783721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:41.783047 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:41.783721 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.783185 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:41.783721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:41.783615 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:41.784003 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:41.783729 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:42.781794 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:42.781756 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:42.782335 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:42.781890 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:43.781724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:43.781633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:43.781904 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:43.781783 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:43.782274 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:43.782182 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:43.782390 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:43.782363 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:44.781482 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:44.781442 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:44.781658 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:44.781592 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:45.782300 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:45.782261 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:45.782713 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:45.782261 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:45.782713 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:45.782409 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:45.782713 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:45.782464 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:46.781441 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:46.781397 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:46.781639 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:46.781527 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:47.782114 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:47.782074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:47.782491 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:47.782231 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:47.782491 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:47.782252 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:47.782491 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:47.782352 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:48.781896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:48.781859 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:48.782133 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:48.781996 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:49.426636 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:49.426583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:49.426907 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.426768 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:49.426907 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.426839 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:05.426820768 +0000 UTC m=+34.124308327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:52:49.527897 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:49.527855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:49.528083 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.528001 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:49.528083 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.528069 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:53:05.528047095 +0000 UTC m=+34.225534655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:52:49.628330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:49.628289 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:49.628513 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.628420 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:52:49.628513 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.628436 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:52:49.628513 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.628444 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:49.628513 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.628491 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:05.628479719 +0000 UTC m=+34.325967276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:52:49.781839 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:49.781750 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:49.782014 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:49.781752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:49.782014 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.781864 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:49.782014 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:49.781960 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:50.781936 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:50.781890 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:50.782398 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:50.782030 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:51.783062 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:51.783019 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:51.783558 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:51.783143 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:51.783558 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:51.783139 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:51.783558 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:51.783243 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:52.781555 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:52.781519 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:52.781756 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:52.781636 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:53.781782 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:53.781741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:53.782383 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:53.781741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:53.782383 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:53.781899 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:53.782383 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:53.781982 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:54.781595 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:54.781285 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:54.781753 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:54.781694 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:55.014520 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.014432 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-54wbd" event={"ID":"ea945cc8-1abb-425d-8230-3b9e1b5c8a26","Type":"ContainerStarted","Data":"64c8d49fdffd6212005695f43d4e3a3d09bf93cc8d9b4063d834534bf7142c46"} Jul 12 21:52:55.015859 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.015833 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wktg5" event={"ID":"4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9","Type":"ContainerStarted","Data":"ff291c8c94f9b47198bd0f5e1a0199d1530e2628e4e852dce5e9f3f7e856b646"} Jul 12 21:52:55.017091 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.017068 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" event={"ID":"6d64f326-fae3-48ef-834f-10d3f9080aaa","Type":"ContainerStarted","Data":"6aca0a332f2adb4860082129380f4ca263fe7e778ab62bea6262f03d4d5c612a"} Jul 12 21:52:55.018330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.018310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rsbww" event={"ID":"1545efb1-e4a2-4ab6-8001-4137a9a081fe","Type":"ContainerStarted","Data":"e18da1da6de64bc5f48814c2bfd7fbbc2995703c161d84619947a3863742d262"} Jul 12 21:52:55.019557 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.019536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nr65r" event={"ID":"48fd652c-b3e0-4244-8b32-c80081d686bc","Type":"ContainerStarted","Data":"a0efcbeba2e32a1b28fc98c8daf0be3720d71378f20a577429b1d3da72a5fbf3"} Jul 12 21:52:55.020861 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.020839 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="9eadf2e6b33082ddcebd844447d2fe3ab9c958bf6bb1b77f609f38cfa3f166d4" exitCode=0 Jul 12 21:52:55.020958 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.020913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"9eadf2e6b33082ddcebd844447d2fe3ab9c958bf6bb1b77f609f38cfa3f166d4"} Jul 12 21:52:55.027147 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027116 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"013edd16ceb9e0ae0bcbc8502b9e7aca3caab9f9dd280dde2cb85ddcb240ff08"} Jul 12 21:52:55.027283 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027265 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"2d9e6381b07b27f1fa5403bea42eb12e30b5298e523cee395a2756f36fd67402"} Jul 12 21:52:55.027358 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027325 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"01d234e85951109a421c6c975ef5d39fc96e1012fe00ce1d850921c07a14bbe4"} Jul 12 21:52:55.027414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027374 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"2e94e722f1d05cd9473f4769fdcde6ba8194dc268daec69bd5359111bd596285"} Jul 12 21:52:55.027467 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027422 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"75f9a7c3fdf988f60b41f9fd34f5d23f4f2a0e8a8d7b13c228e62c67aee1d76b"} Jul 12 21:52:55.027519 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.027472 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"d3e418270c7bda674595da0c7d6cf5928daddbbe5738145c6f3e2b1be439444c"} Jul 12 21:52:55.028470 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.028449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7bnnm" event={"ID":"2f9b93ec-4582-4341-be8a-e777ff69340c","Type":"ContainerStarted","Data":"53d8a01636387809c07ae7224f416961a30a950b53a7acb0a25ca2184e77f346"} Jul 12 21:52:55.047585 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.047535 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-6.ec2.internal" podStartSLOduration=22.047519986 podStartE2EDuration="22.047519986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:36.995568125 +0000 UTC m=+5.693055709" watchObservedRunningTime="2026-07-12 21:52:55.047519986 +0000 UTC m=+23.745007566" Jul 12 21:52:55.099090 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.099036 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-54wbd" podStartSLOduration=3.855887257 podStartE2EDuration="23.099018127s" totalImagesPullingTime="19.24313087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:55.047484627 +0000 UTC m=+23.744972207" watchObservedRunningTime="2026-07-12 21:52:55.099018127 +0000 UTC m=+23.796505721" Jul 12 21:52:55.147868 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.147817 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7bnnm" podStartSLOduration=3.876538274 podStartE2EDuration="23.147801878s" totalImagesPullingTime="19.271263604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:55.099997176 +0000 UTC m=+23.797484767" watchObservedRunningTime="2026-07-12 21:52:55.147801878 +0000 UTC m=+23.845289457" Jul 12 21:52:55.171418 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.171370 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nr65r" podStartSLOduration=3.957158381 podStartE2EDuration="23.171353741s" totalImagesPullingTime="19.21419536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:55.171092461 +0000 UTC m=+23.868580041" watchObservedRunningTime="2026-07-12 21:52:55.171353741 +0000 UTC m=+23.868841322" Jul 12 21:52:55.199633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.199581 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wktg5" podStartSLOduration=3.977041449 podStartE2EDuration="23.199564337s" totalImagesPullingTime="19.222522888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:55.199195249 +0000 UTC m=+23.896682829" watchObservedRunningTime="2026-07-12 21:52:55.199564337 +0000 UTC m=+23.897051973" Jul 12 21:52:55.257583 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.257530 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rsbww" podStartSLOduration=4.033150825 podStartE2EDuration="23.257512196s" totalImagesPullingTime="19.224361371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:55.257420524 +0000 UTC m=+23.954908104" watchObservedRunningTime="2026-07-12 21:52:55.257512196 +0000 UTC m=+23.954999775" Jul 12 21:52:55.439445 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.439417 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 21:52:55.757038 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.756859 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T21:52:55.439436573Z","UUID":"2ccc5fd8-3960-4798-85ff-751f9268dee4","Handler":null,"Name":"","Endpoint":""} Jul 12 21:52:55.758510 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.758482 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 21:52:55.758510 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.758516 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 21:52:55.781781 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.781750 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:55.781983 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:55.781750 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:55.781983 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:55.781872 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:55.782112 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:55.781990 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:56.033284 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.033183 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" event={"ID":"6d64f326-fae3-48ef-834f-10d3f9080aaa","Type":"ContainerStarted","Data":"13224db2d38ec65a7caa35ad9df36c1f4e7f56f201b7381046ec886de44830c9"} Jul 12 21:52:56.035102 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.035060 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6pcwz" event={"ID":"cd8d6b7a-3815-4786-91f8-d11babe079dd","Type":"ContainerStarted","Data":"64765c0a70e1f10b41d06f610f08048b6fccd47d094fac7ddbcbd831eea22fdd"} Jul 12 21:52:56.060730 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.060679 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6pcwz" podStartSLOduration=4.835379987 podStartE2EDuration="24.060657274s" totalImagesPullingTime="19.225277287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:56.059131664 +0000 UTC m=+24.756619243" watchObservedRunningTime="2026-07-12 21:52:56.060657274 +0000 UTC m=+24.758144853" Jul 12 21:52:56.782378 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.782297 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:56.782536 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:56.782435 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:56.975788 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.975751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:56.976418 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:56.976388 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:52:57.041370 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:57.041287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"5608aaf18f118450225b2f0a70f0d00b051538284a2bfc7d1e1573985a77849f"} Jul 12 21:52:57.043907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:57.043864 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" event={"ID":"6d64f326-fae3-48ef-834f-10d3f9080aaa","Type":"ContainerStarted","Data":"ec8c4e6a700e71dc84d60dd072c6b1f2f23406a672a7ae699a50d293396f2ef0"} Jul 12 21:52:57.067961 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:57.067901 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnn87" podStartSLOduration=3.687066031 podStartE2EDuration="25.067884358s" totalImagesPullingTime="21.380818327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:52:57.067718749 +0000 UTC m=+25.765206330" watchObservedRunningTime="2026-07-12 21:52:57.067884358 +0000 UTC m=+25.765371937" Jul 12 21:52:57.781994 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:57.781963 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:57.782199 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:57.781963 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:57.782199 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:57.782125 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:52:57.782199 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:57.782168 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:58.046492 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:58.046160 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 21:52:58.782269 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:58.782235 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:52:58.782508 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:58.782358 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:52:59.781831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:59.781545 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:52:59.781831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:52:59.781604 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:52:59.782629 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:59.781852 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:52:59.782629 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:52:59.781912 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:53:00.057164 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.057072 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="6b54f5e96102391b20c117deaea54390993fc16aa6054f0e16b1290d72e5e586" exitCode=0 Jul 12 21:53:00.057164 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.057113 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"6b54f5e96102391b20c117deaea54390993fc16aa6054f0e16b1290d72e5e586"} Jul 12 21:53:00.061197 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.061165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" event={"ID":"1c58b8f6-f0c0-4142-86d0-de5049ec66b5","Type":"ContainerStarted","Data":"1951534c8cb3fbd6b7fc13f666873907fd0db8e2909ea2c3f62cbcdf35f16824"} Jul 12 21:53:00.061465 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.061439 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:00.077798 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.077769 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:00.114870 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.114814 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" podStartSLOduration=8.823550301000001 podStartE2EDuration="28.114797441s" totalImagesPullingTime="19.29124714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:00.114475549 +0000 UTC m=+28.811963129" watchObservedRunningTime="2026-07-12 21:53:00.114797441 +0000 UTC m=+28.812285021" Jul 12 21:53:00.781809 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:00.781771 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:00.782064 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:00.781901 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:53:01.063659 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.063591 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:01.063832 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.063661 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:01.079493 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.079463 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:01.148333 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.148302 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c7l4v"] Jul 12 21:53:01.148495 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.148450 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:01.148580 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:01.148543 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:53:01.153753 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.153722 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-st2cg"] Jul 12 21:53:01.153942 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.153894 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:01.154081 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:01.154044 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:53:01.154419 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.154400 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mtcjn"] Jul 12 21:53:01.154506 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.154497 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:01.154637 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:01.154581 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:53:01.523134 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.523035 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:53:01.523360 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.523177 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 21:53:01.523776 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:01.523758 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nr65r" Jul 12 21:53:02.067243 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:02.067189 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="3c8e4655807f456cc4973e3a80cf5838995d872af19bee45bfd8b263f4a2f29a" exitCode=0 Jul 12 21:53:02.067633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:02.067244 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"3c8e4655807f456cc4973e3a80cf5838995d872af19bee45bfd8b263f4a2f29a"} Jul 12 21:53:02.782428 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:02.782243 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:02.782630 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:02.782243 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:02.782630 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:02.782516 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:53:02.782630 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:02.782266 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:02.782798 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:02.782583 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:53:02.782798 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:02.782751 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:53:04.073870 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:04.073828 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="229acf4510db230f58e5c11fa7f4a28f0fe9914454a20cb355b3e4d54f6f9d1b" exitCode=0 Jul 12 21:53:04.074276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:04.073901 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"229acf4510db230f58e5c11fa7f4a28f0fe9914454a20cb355b3e4d54f6f9d1b"} Jul 12 21:53:04.782275 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:04.782238 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:04.782275 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:04.782280 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:04.782534 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:04.782252 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:04.782534 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:04.782389 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:53:04.782534 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:04.782499 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:53:04.782653 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:04.782571 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:53:05.442896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:05.442858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:05.443523 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.443041 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 21:53:05.443523 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.443126 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret podName:fee60708-57c1-4b5d-97b6-e5ea3ed089c1 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:37.443107083 +0000 UTC m=+66.140594641 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret") pod "global-pull-secret-syncer-c7l4v" (UID: "fee60708-57c1-4b5d-97b6-e5ea3ed089c1") : object "kube-system"/"original-pull-secret" not registered Jul 12 21:53:05.543860 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:05.543826 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:05.544029 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.543984 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:53:05.544095 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.544061 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs podName:4841a61b-e23a-4fab-ab80-6705746aaf9a nodeName:}" failed. No retries permitted until 2026-07-12 21:53:37.544039082 +0000 UTC m=+66.241526643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs") pod "network-metrics-daemon-mtcjn" (UID: "4841a61b-e23a-4fab-ab80-6705746aaf9a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 21:53:05.644987 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:05.644954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:05.645186 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.645112 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 21:53:05.645186 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.645132 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 21:53:05.645186 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.645143 2575 projected.go:196] Error preparing data for projected volume kube-api-access-ptqtp for pod openshift-network-diagnostics/network-check-target-st2cg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:53:05.645374 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:05.645233 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp podName:7b709736-c734-4679-b2d4-2388270e8d67 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:37.645181172 +0000 UTC m=+66.342668731 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ptqtp" (UniqueName: "kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp") pod "network-check-target-st2cg" (UID: "7b709736-c734-4679-b2d4-2388270e8d67") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 21:53:06.781826 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:06.781791 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:06.781826 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:06.781819 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:06.782527 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:06.781912 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st2cg" podUID="7b709736-c734-4679-b2d4-2388270e8d67" Jul 12 21:53:06.782527 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:06.781967 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:06.782527 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:06.782090 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7l4v" podUID="fee60708-57c1-4b5d-97b6-e5ea3ed089c1" Jul 12 21:53:06.782527 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:06.782195 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mtcjn" podUID="4841a61b-e23a-4fab-ab80-6705746aaf9a" Jul 12 21:53:07.596733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.596700 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-6.ec2.internal" event="NodeReady" Jul 12 21:53:07.596910 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.596853 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 21:53:07.643302 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.643262 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lmw5c"] Jul 12 21:53:07.647668 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.646618 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.648290 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.647891 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tvsxt"] Jul 12 21:53:07.650022 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.649992 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6dkr4\"" Jul 12 21:53:07.650817 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.649992 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 21:53:07.650817 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.650528 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.650817 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.650605 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 21:53:07.653592 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.653566 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 21:53:07.653733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.653648 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 21:53:07.653733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.653677 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 21:53:07.653733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.653720 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2rf25\"" Jul 12 21:53:07.661616 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.661086 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lmw5c"] Jul 12 21:53:07.664008 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.663979 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tvsxt"] Jul 12 21:53:07.760701 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc2mx\" (UniqueName: \"kubernetes.io/projected/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-kube-api-access-gc2mx\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.760896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm4rh\" (UniqueName: \"kubernetes.io/projected/651f2cc2-fa5e-482c-864e-da1f52675d9d-kube-api-access-vm4rh\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.760896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.760896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-config-volume\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.760896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760875 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-tmp-dir\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.760896 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.760897 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/651f2cc2-fa5e-482c-864e-da1f52675d9d-cert\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.861706 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm4rh\" (UniqueName: \"kubernetes.io/projected/651f2cc2-fa5e-482c-864e-da1f52675d9d-kube-api-access-vm4rh\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.861706 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861721 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-config-volume\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-tmp-dir\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/651f2cc2-fa5e-482c-864e-da1f52675d9d-cert\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.861771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc2mx\" (UniqueName: \"kubernetes.io/projected/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-kube-api-access-gc2mx\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:07.861846 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:07.861913 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls podName:031e3ac8-87ac-4993-a35f-f0f016f1d0c4 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:08.36189262 +0000 UTC m=+37.059380178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls") pod "dns-default-lmw5c" (UID: "031e3ac8-87ac-4993-a35f-f0f016f1d0c4") : secret "dns-default-metrics-tls" not found Jul 12 21:53:07.862431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.862179 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-tmp-dir\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.862765 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.862473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-config-volume\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.866731 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.866702 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/651f2cc2-fa5e-482c-864e-da1f52675d9d-cert\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.873761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.873732 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm4rh\" (UniqueName: \"kubernetes.io/projected/651f2cc2-fa5e-482c-864e-da1f52675d9d-kube-api-access-vm4rh\") pod \"ingress-canary-tvsxt\" (UID: \"651f2cc2-fa5e-482c-864e-da1f52675d9d\") " pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:07.873761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.873756 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc2mx\" (UniqueName: \"kubernetes.io/projected/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-kube-api-access-gc2mx\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:07.968080 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:07.968035 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tvsxt" Jul 12 21:53:08.160559 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.160383 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tvsxt"] Jul 12 21:53:08.163999 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:08.163969 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651f2cc2_fa5e_482c_864e_da1f52675d9d.slice/crio-7ccff86daed1267bd4efc8121b0031f35066a83f9d5f5ba54f9f53bc20fbd6d8 WatchSource:0}: Error finding container 7ccff86daed1267bd4efc8121b0031f35066a83f9d5f5ba54f9f53bc20fbd6d8: Status 404 returned error can't find the container with id 7ccff86daed1267bd4efc8121b0031f35066a83f9d5f5ba54f9f53bc20fbd6d8 Jul 12 21:53:08.365495 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.365450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:08.365676 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:08.365605 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 21:53:08.365721 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:08.365684 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls podName:031e3ac8-87ac-4993-a35f-f0f016f1d0c4 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:09.365665412 +0000 UTC m=+38.063152985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls") pod "dns-default-lmw5c" (UID: "031e3ac8-87ac-4993-a35f-f0f016f1d0c4") : secret "dns-default-metrics-tls" not found Jul 12 21:53:08.782344 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.782283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:08.782344 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.782318 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:08.782598 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.782282 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:08.785309 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.785282 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 21:53:08.785477 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.785368 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 21:53:08.786510 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.786488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 21:53:08.786510 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.786490 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 21:53:08.786678 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.786615 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4x2vl\"" Jul 12 21:53:08.786735 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:08.786716 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j98hn\"" Jul 12 21:53:09.089891 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.089807 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="68915328c5c540599f0a6c4fc01a3a9c9fe5f02f18bceb5d4cdeb035db976d18" exitCode=0 Jul 12 21:53:09.090475 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.089900 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"68915328c5c540599f0a6c4fc01a3a9c9fe5f02f18bceb5d4cdeb035db976d18"} Jul 12 21:53:09.091287 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.091253 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tvsxt" event={"ID":"651f2cc2-fa5e-482c-864e-da1f52675d9d","Type":"ContainerStarted","Data":"7ccff86daed1267bd4efc8121b0031f35066a83f9d5f5ba54f9f53bc20fbd6d8"} Jul 12 21:53:09.375824 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.375733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:09.376036 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:09.375899 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 21:53:09.376036 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:09.375976 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls podName:031e3ac8-87ac-4993-a35f-f0f016f1d0c4 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:11.375953452 +0000 UTC m=+40.073441011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls") pod "dns-default-lmw5c" (UID: "031e3ac8-87ac-4993-a35f-f0f016f1d0c4") : secret "dns-default-metrics-tls" not found Jul 12 21:53:09.987953 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.987919 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4lxsf"] Jul 12 21:53:09.992532 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.992496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:09.996107 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.996078 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 21:53:09.996996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.996805 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 21:53:09.996996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.996844 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 21:53:09.996996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.996874 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwml9\"" Jul 12 21:53:09.996996 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:09.996954 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 21:53:10.006931 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.006892 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4lxsf"] Jul 12 21:53:10.082979 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.082946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.083166 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.083024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-crio-socket\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.083166 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.083103 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-data-volume\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.083166 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.083129 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pr6r\" (UniqueName: \"kubernetes.io/projected/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-api-access-4pr6r\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.083332 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.083230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.097882 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.097843 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d4a260a-963c-4a9d-938b-29ec4d903536" containerID="ee738ea7446fd38e5c2643291604221df1924768a34c6e44dea33ec7543eae45" exitCode=0 Jul 12 21:53:10.098369 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.097903 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerDied","Data":"ee738ea7446fd38e5c2643291604221df1924768a34c6e44dea33ec7543eae45"} Jul 12 21:53:10.183902 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.183832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-data-volume\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.183902 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.183892 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pr6r\" (UniqueName: \"kubernetes.io/projected/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-api-access-4pr6r\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.184152 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.183948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.184152 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.184006 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.184152 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.184131 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 21:53:10.184331 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.184149 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:10.184331 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.184232 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:10.684189392 +0000 UTC m=+39.381676960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : configmap "kube-rbac-proxy" not found Jul 12 21:53:10.184331 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.184227 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-data-volume\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.184331 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.184248 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:10.68424134 +0000 UTC m=+39.381728898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:10.184331 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.184317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-crio-socket\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.184578 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.184535 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-crio-socket\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.196377 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.196325 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pr6r\" (UniqueName: \"kubernetes.io/projected/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-api-access-4pr6r\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.372047 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.371858 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt"] Jul 12 21:53:10.380831 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.380803 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" Jul 12 21:53:10.384034 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.383763 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 21:53:10.384034 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.383922 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 21:53:10.384651 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.384629 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt"] Jul 12 21:53:10.385317 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.385296 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-44kxs\"" Jul 12 21:53:10.486177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.486149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkfb8\" (UniqueName: \"kubernetes.io/projected/565820fb-1c32-4a23-a236-12adac158923-kube-api-access-hkfb8\") pod \"migrator-7f59b54f6c-694pt\" (UID: \"565820fb-1c32-4a23-a236-12adac158923\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" Jul 12 21:53:10.587699 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.587613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkfb8\" (UniqueName: \"kubernetes.io/projected/565820fb-1c32-4a23-a236-12adac158923-kube-api-access-hkfb8\") pod \"migrator-7f59b54f6c-694pt\" (UID: \"565820fb-1c32-4a23-a236-12adac158923\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" Jul 12 21:53:10.597472 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.597437 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkfb8\" (UniqueName: \"kubernetes.io/projected/565820fb-1c32-4a23-a236-12adac158923-kube-api-access-hkfb8\") pod \"migrator-7f59b54f6c-694pt\" (UID: \"565820fb-1c32-4a23-a236-12adac158923\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" Jul 12 21:53:10.688383 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.688346 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.688548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.688400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:10.688548 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.688504 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:10.688627 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.688571 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:11.688555822 +0000 UTC m=+40.386043380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:10.688627 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.688512 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 21:53:10.688627 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:10.688605 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:11.68859898 +0000 UTC m=+40.386086538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : configmap "kube-rbac-proxy" not found Jul 12 21:53:10.691360 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.691320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" Jul 12 21:53:10.859540 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:10.859455 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt"] Jul 12 21:53:10.862978 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:10.862944 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565820fb_1c32_4a23_a236_12adac158923.slice/crio-84f539765989520c4572c5ac2378427ea3e44f14c49f1473d71edf61199800a7 WatchSource:0}: Error finding container 84f539765989520c4572c5ac2378427ea3e44f14c49f1473d71edf61199800a7: Status 404 returned error can't find the container with id 84f539765989520c4572c5ac2378427ea3e44f14c49f1473d71edf61199800a7 Jul 12 21:53:11.103288 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.103250 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" event={"ID":"565820fb-1c32-4a23-a236-12adac158923","Type":"ContainerStarted","Data":"84f539765989520c4572c5ac2378427ea3e44f14c49f1473d71edf61199800a7"} Jul 12 21:53:11.106654 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.106615 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" event={"ID":"7d4a260a-963c-4a9d-938b-29ec4d903536","Type":"ContainerStarted","Data":"87427276ef141f9191e7f8ea711b5526580616d30a8ec87521a420f7755eabde"} Jul 12 21:53:11.108058 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.108032 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tvsxt" event={"ID":"651f2cc2-fa5e-482c-864e-da1f52675d9d","Type":"ContainerStarted","Data":"d2622accd72356fb1e8de875285c4e9e7c717a98a3271cd5eb529fbce803a959"} Jul 12 21:53:11.133526 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.133457 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bzrhw" podStartSLOduration=9.610522531 podStartE2EDuration="39.133433797s" totalImagesPullingTime="29.522911266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:11.131309967 +0000 UTC m=+39.828797547" watchObservedRunningTime="2026-07-12 21:53:11.133433797 +0000 UTC m=+39.830921377" Jul 12 21:53:11.149262 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.149189 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tvsxt" podStartSLOduration=1.8995055939999999 podStartE2EDuration="4.149175714s" totalImagesPullingTime="2.24967012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:11.148246871 +0000 UTC m=+39.845734452" watchObservedRunningTime="2026-07-12 21:53:11.149175714 +0000 UTC m=+39.846663293" Jul 12 21:53:11.394914 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.394824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:11.395061 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.394967 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 21:53:11.395061 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.395023 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls podName:031e3ac8-87ac-4993-a35f-f0f016f1d0c4 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:15.395008065 +0000 UTC m=+44.092495626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls") pod "dns-default-lmw5c" (UID: "031e3ac8-87ac-4993-a35f-f0f016f1d0c4") : secret "dns-default-metrics-tls" not found Jul 12 21:53:11.696863 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.696763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:11.696863 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:11.696831 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:11.697097 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.696956 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:11.697097 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.697042 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:13.697016546 +0000 UTC m=+42.394504109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:11.697097 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.696956 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 21:53:11.697275 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:11.697132 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:13.697110557 +0000 UTC m=+42.394598118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : configmap "kube-rbac-proxy" not found Jul 12 21:53:12.141754 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.141568 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rsbww_1545efb1-e4a2-4ab6-8001-4137a9a081fe/dns-node-resolver/0.log" Jul 12 21:53:12.393815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.393734 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rdcwx"] Jul 12 21:53:12.410339 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.410302 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rdcwx"] Jul 12 21:53:12.410506 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.410446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.414467 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.414141 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vvvh4\"" Jul 12 21:53:12.414467 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.414330 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 21:53:12.414467 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.414416 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 21:53:12.416976 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.415009 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 21:53:12.416976 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.415239 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 21:53:12.502447 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.502417 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-cabundle\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.502591 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.502453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28r7\" (UniqueName: \"kubernetes.io/projected/15b046f1-db03-44bb-a1fd-da39ac2bee26-kube-api-access-w28r7\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.502591 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.502561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-key\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.603930 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.603894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-key\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.604087 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.604001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-cabundle\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.604087 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.604031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28r7\" (UniqueName: \"kubernetes.io/projected/15b046f1-db03-44bb-a1fd-da39ac2bee26-kube-api-access-w28r7\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.604769 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.604740 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-cabundle\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.607625 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.607605 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/15b046f1-db03-44bb-a1fd-da39ac2bee26-signing-key\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.620316 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.620288 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28r7\" (UniqueName: \"kubernetes.io/projected/15b046f1-db03-44bb-a1fd-da39ac2bee26-kube-api-access-w28r7\") pod \"service-ca-79f68b9f4f-rdcwx\" (UID: \"15b046f1-db03-44bb-a1fd-da39ac2bee26\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.724126 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.724085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" Jul 12 21:53:12.880190 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:12.880152 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rdcwx"] Jul 12 21:53:12.883984 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:12.883951 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b046f1_db03_44bb_a1fd_da39ac2bee26.slice/crio-30d00846317c8fbf39fe45d36d5b894eae7e8c81dcf8ac0b6587fed746a1cc69 WatchSource:0}: Error finding container 30d00846317c8fbf39fe45d36d5b894eae7e8c81dcf8ac0b6587fed746a1cc69: Status 404 returned error can't find the container with id 30d00846317c8fbf39fe45d36d5b894eae7e8c81dcf8ac0b6587fed746a1cc69 Jul 12 21:53:13.115021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.114928 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" event={"ID":"565820fb-1c32-4a23-a236-12adac158923","Type":"ContainerStarted","Data":"269ff812ea57040e7ec25e7dafb50ffc81e187fc9ee55c809108db071140029a"} Jul 12 21:53:13.115021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.114994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" event={"ID":"565820fb-1c32-4a23-a236-12adac158923","Type":"ContainerStarted","Data":"f1b0ce5a54cdd0e8cbec0d82550cafc627559a057a9636d8cefe085eb063a6b2"} Jul 12 21:53:13.116103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.116078 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" event={"ID":"15b046f1-db03-44bb-a1fd-da39ac2bee26","Type":"ContainerStarted","Data":"30d00846317c8fbf39fe45d36d5b894eae7e8c81dcf8ac0b6587fed746a1cc69"} Jul 12 21:53:13.141620 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.141568 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-694pt" podStartSLOduration=1.509092136 podStartE2EDuration="3.141553393s" totalImagesPullingTime="1.632461257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:13.14118906 +0000 UTC m=+41.838676641" watchObservedRunningTime="2026-07-12 21:53:13.141553393 +0000 UTC m=+41.839040951" Jul 12 21:53:13.159352 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.159324 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wktg5_4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9/node-ca/0.log" Jul 12 21:53:13.712846 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.712812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:13.713047 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.712915 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:13.713047 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:13.712950 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 21:53:13.713047 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:13.713019 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:17.713000904 +0000 UTC m=+46.410488475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : configmap "kube-rbac-proxy" not found Jul 12 21:53:13.713047 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:13.713024 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:13.713292 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:13.713055 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:17.713047857 +0000 UTC m=+46.410535416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:13.946452 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:13.946425 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tvsxt_651f2cc2-fa5e-482c-864e-da1f52675d9d/serve-healthcheck-canary/0.log" Jul 12 21:53:15.428020 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:15.427981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:15.428425 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:15.428158 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 21:53:15.428425 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:15.428253 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls podName:031e3ac8-87ac-4993-a35f-f0f016f1d0c4 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:23.42823219 +0000 UTC m=+52.125719753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls") pod "dns-default-lmw5c" (UID: "031e3ac8-87ac-4993-a35f-f0f016f1d0c4") : secret "dns-default-metrics-tls" not found Jul 12 21:53:16.124519 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:16.124479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" event={"ID":"15b046f1-db03-44bb-a1fd-da39ac2bee26","Type":"ContainerStarted","Data":"42afd8d100ac431c6964ad61fa933e5d75add461956731ecf58bfcb8b4afdeee"} Jul 12 21:53:16.144591 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:16.144536 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-rdcwx" podStartSLOduration=1.953481498 podStartE2EDuration="4.144517706s" totalImagesPullingTime="2.191036208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:16.144323175 +0000 UTC m=+44.841810756" watchObservedRunningTime="2026-07-12 21:53:16.144517706 +0000 UTC m=+44.842005286" Jul 12 21:53:17.746547 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:17.746498 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:17.746547 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:17.746540 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:17.747042 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:17.746586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:17.747042 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:17.746602 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:25.746586562 +0000 UTC m=+54.444074120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:17.747042 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:17.746637 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 21:53:17.747042 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:17.746685 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:25.74667142 +0000 UTC m=+54.444159000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : configmap "kube-rbac-proxy" not found Jul 12 21:53:23.493462 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:23.493418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:23.495720 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:23.495694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/031e3ac8-87ac-4993-a35f-f0f016f1d0c4-metrics-tls\") pod \"dns-default-lmw5c\" (UID: \"031e3ac8-87ac-4993-a35f-f0f016f1d0c4\") " pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:23.561128 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:23.561084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:23.712656 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:23.712626 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lmw5c"] Jul 12 21:53:23.716588 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:23.716558 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031e3ac8_87ac_4993_a35f_f0f016f1d0c4.slice/crio-dd8071f07b57301e66d4ce203c3b37c95224ef224091bcf27a0530026fb92515 WatchSource:0}: Error finding container dd8071f07b57301e66d4ce203c3b37c95224ef224091bcf27a0530026fb92515: Status 404 returned error can't find the container with id dd8071f07b57301e66d4ce203c3b37c95224ef224091bcf27a0530026fb92515 Jul 12 21:53:24.145686 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:24.145647 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lmw5c" event={"ID":"031e3ac8-87ac-4993-a35f-f0f016f1d0c4","Type":"ContainerStarted","Data":"dd8071f07b57301e66d4ce203c3b37c95224ef224091bcf27a0530026fb92515"} Jul 12 21:53:25.807433 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:25.807388 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:25.807815 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:25.807454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:25.807815 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:25.807541 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 21:53:25.807815 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:25.807608 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls podName:6ac3cb4e-fb16-4f25-a377-bb22b67ad865 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:41.807592599 +0000 UTC m=+70.505080161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4lxsf" (UID: "6ac3cb4e-fb16-4f25-a377-bb22b67ad865") : secret "insights-runtime-extractor-tls" not found Jul 12 21:53:25.807923 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:25.807893 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:26.154885 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:26.154843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lmw5c" event={"ID":"031e3ac8-87ac-4993-a35f-f0f016f1d0c4","Type":"ContainerStarted","Data":"07d915b457491aa760d8431c64fbca1a70fd968bbb51d30d0201b524cb4c371b"} Jul 12 21:53:27.159756 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:27.159719 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lmw5c" event={"ID":"031e3ac8-87ac-4993-a35f-f0f016f1d0c4","Type":"ContainerStarted","Data":"1a49f3d3266575bf3b26c9ce7fc9457ee57915b57c6fb960235b6800a9301d6d"} Jul 12 21:53:27.160263 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:27.159827 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:27.179190 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:27.179134 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lmw5c" podStartSLOduration=17.919991695 podStartE2EDuration="20.179114353s" totalImagesPullingTime="2.259122658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:27.178868902 +0000 UTC m=+55.876356482" watchObservedRunningTime="2026-07-12 21:53:27.179114353 +0000 UTC m=+55.876601933" Jul 12 21:53:33.088667 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:33.088638 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzvt7" Jul 12 21:53:35.825912 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.825870 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-d8vlr"] Jul 12 21:53:35.828709 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.828686 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:35.832969 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.832948 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cpmzn\"" Jul 12 21:53:35.833352 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.833334 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 21:53:35.833412 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.833370 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 21:53:35.848478 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.848442 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-d8vlr"] Jul 12 21:53:35.872680 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.872649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxb8m\" (UniqueName: \"kubernetes.io/projected/3a74edf0-42f2-4bad-b978-6f91498803cd-kube-api-access-gxb8m\") pod \"downloads-68c9cbb497-d8vlr\" (UID: \"3a74edf0-42f2-4bad-b978-6f91498803cd\") " pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:35.929532 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.929497 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fd87957d-4h4ld"] Jul 12 21:53:35.931600 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.931582 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.934323 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.934299 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jvnd\"" Jul 12 21:53:35.934448 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.934326 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 21:53:35.934595 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.934580 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 21:53:35.934794 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.934780 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 21:53:35.940067 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.940042 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 21:53:35.945925 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.945890 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd87957d-4h4ld"] Jul 12 21:53:35.973497 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-image-registry-private-configuration\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973512 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxb8m\" (UniqueName: \"kubernetes.io/projected/3a74edf0-42f2-4bad-b978-6f91498803cd-kube-api-access-gxb8m\") pod \"downloads-68c9cbb497-d8vlr\" (UID: \"3a74edf0-42f2-4bad-b978-6f91498803cd\") " pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-trusted-ca\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973556 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-installation-pull-secrets\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973582 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cf50fc4-2925-451b-b448-80bbc82992c0-ca-trust-extracted\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-certificates\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973668 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-bound-sa-token\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973693 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-tls\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.973707 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.973709 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzfk9\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-kube-api-access-lzfk9\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:35.988921 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:35.988881 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxb8m\" (UniqueName: \"kubernetes.io/projected/3a74edf0-42f2-4bad-b978-6f91498803cd-kube-api-access-gxb8m\") pod \"downloads-68c9cbb497-d8vlr\" (UID: \"3a74edf0-42f2-4bad-b978-6f91498803cd\") " pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:36.074743 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.074702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-certificates\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.074951 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.074758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-bound-sa-token\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.074951 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.074882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-tls\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.074951 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.074927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzfk9\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-kube-api-access-lzfk9\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075134 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-image-registry-private-configuration\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075134 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-trusted-ca\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075134 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-installation-pull-secrets\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075134 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075121 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cf50fc4-2925-451b-b448-80bbc82992c0-ca-trust-extracted\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075577 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cf50fc4-2925-451b-b448-80bbc82992c0-ca-trust-extracted\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.075733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.075705 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-certificates\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.076190 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.076134 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cf50fc4-2925-451b-b448-80bbc82992c0-trusted-ca\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.077559 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.077537 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-image-registry-private-configuration\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.077647 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.077606 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-registry-tls\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.077733 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.077708 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cf50fc4-2925-451b-b448-80bbc82992c0-installation-pull-secrets\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.093113 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.093083 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-bound-sa-token\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.093288 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.093274 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzfk9\" (UniqueName: \"kubernetes.io/projected/4cf50fc4-2925-451b-b448-80bbc82992c0-kube-api-access-lzfk9\") pod \"image-registry-fd87957d-4h4ld\" (UID: \"4cf50fc4-2925-451b-b448-80bbc82992c0\") " pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.137417 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.137385 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:36.241651 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.241620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:36.272600 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.272562 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-d8vlr"] Jul 12 21:53:36.276988 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:36.276960 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a74edf0_42f2_4bad_b978_6f91498803cd.slice/crio-a4b5b01110d77cd13c528ef9ed41e365ffe83c0b9ebb5aabd1632dc868113053 WatchSource:0}: Error finding container a4b5b01110d77cd13c528ef9ed41e365ffe83c0b9ebb5aabd1632dc868113053: Status 404 returned error can't find the container with id a4b5b01110d77cd13c528ef9ed41e365ffe83c0b9ebb5aabd1632dc868113053 Jul 12 21:53:36.379585 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:36.379499 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd87957d-4h4ld"] Jul 12 21:53:36.383194 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:36.383166 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf50fc4_2925_451b_b448_80bbc82992c0.slice/crio-e16b7984fd4160819d100382bd7b31370647685e1fdba98fb73046f664b23c61 WatchSource:0}: Error finding container e16b7984fd4160819d100382bd7b31370647685e1fdba98fb73046f664b23c61: Status 404 returned error can't find the container with id e16b7984fd4160819d100382bd7b31370647685e1fdba98fb73046f664b23c61 Jul 12 21:53:37.165374 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.165338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lmw5c" Jul 12 21:53:37.196197 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.196155 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-d8vlr" event={"ID":"3a74edf0-42f2-4bad-b978-6f91498803cd","Type":"ContainerStarted","Data":"a4b5b01110d77cd13c528ef9ed41e365ffe83c0b9ebb5aabd1632dc868113053"} Jul 12 21:53:37.197841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.197803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" event={"ID":"4cf50fc4-2925-451b-b448-80bbc82992c0","Type":"ContainerStarted","Data":"f7ff361ceb30b4f5262f14dc15b17f08b53b559e2306fbacb411eece7cfca311"} Jul 12 21:53:37.198001 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.197858 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" event={"ID":"4cf50fc4-2925-451b-b448-80bbc82992c0","Type":"ContainerStarted","Data":"e16b7984fd4160819d100382bd7b31370647685e1fdba98fb73046f664b23c61"} Jul 12 21:53:37.198001 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.197948 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:37.257292 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.257232 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" podStartSLOduration=2.257198538 podStartE2EDuration="2.257198538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:37.256700104 +0000 UTC m=+65.954187684" watchObservedRunningTime="2026-07-12 21:53:37.257198538 +0000 UTC m=+65.954686155" Jul 12 21:53:37.488982 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.488894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:37.492184 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.492159 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 21:53:37.501966 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.501934 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fee60708-57c1-4b5d-97b6-e5ea3ed089c1-original-pull-secret\") pod \"global-pull-secret-syncer-c7l4v\" (UID: \"fee60708-57c1-4b5d-97b6-e5ea3ed089c1\") " pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:37.589481 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.589437 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:37.592486 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.592460 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 21:53:37.594499 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.594476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7l4v" Jul 12 21:53:37.602142 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.602116 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4841a61b-e23a-4fab-ab80-6705746aaf9a-metrics-certs\") pod \"network-metrics-daemon-mtcjn\" (UID: \"4841a61b-e23a-4fab-ab80-6705746aaf9a\") " pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:37.610794 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.610766 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4x2vl\"" Jul 12 21:53:37.619090 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.619061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mtcjn" Jul 12 21:53:37.692886 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.691766 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:37.695247 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.694913 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 21:53:37.704790 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.704760 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 21:53:37.716689 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.716623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptqtp\" (UniqueName: \"kubernetes.io/projected/7b709736-c734-4679-b2d4-2388270e8d67-kube-api-access-ptqtp\") pod \"network-check-target-st2cg\" (UID: \"7b709736-c734-4679-b2d4-2388270e8d67\") " pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:37.753903 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.753810 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c7l4v"] Jul 12 21:53:37.757532 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:37.757493 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee60708_57c1_4b5d_97b6_e5ea3ed089c1.slice/crio-3541ef459e5b93d6946ce5f0fe42b1a2296599d5fd742956bf2bafd28025153b WatchSource:0}: Error finding container 3541ef459e5b93d6946ce5f0fe42b1a2296599d5fd742956bf2bafd28025153b: Status 404 returned error can't find the container with id 3541ef459e5b93d6946ce5f0fe42b1a2296599d5fd742956bf2bafd28025153b Jul 12 21:53:37.786281 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.786236 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mtcjn"] Jul 12 21:53:37.789185 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:37.789156 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4841a61b_e23a_4fab_ab80_6705746aaf9a.slice/crio-a355af96daddd690316490d853babfa617813b78550ebc16d40cd91f3c9bb97c WatchSource:0}: Error finding container a355af96daddd690316490d853babfa617813b78550ebc16d40cd91f3c9bb97c: Status 404 returned error can't find the container with id a355af96daddd690316490d853babfa617813b78550ebc16d40cd91f3c9bb97c Jul 12 21:53:37.905013 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.904979 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j98hn\"" Jul 12 21:53:37.913385 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:37.913341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:38.076152 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:38.076117 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-st2cg"] Jul 12 21:53:38.080408 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:38.080376 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b709736_c734_4679_b2d4_2388270e8d67.slice/crio-c1406c0b97c2efb077353c18ea23d703b80bc79e97657c4dbe2fb35a932b315e WatchSource:0}: Error finding container c1406c0b97c2efb077353c18ea23d703b80bc79e97657c4dbe2fb35a932b315e: Status 404 returned error can't find the container with id c1406c0b97c2efb077353c18ea23d703b80bc79e97657c4dbe2fb35a932b315e Jul 12 21:53:38.203414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:38.203346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-st2cg" event={"ID":"7b709736-c734-4679-b2d4-2388270e8d67","Type":"ContainerStarted","Data":"c1406c0b97c2efb077353c18ea23d703b80bc79e97657c4dbe2fb35a932b315e"} Jul 12 21:53:38.204902 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:38.204870 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c7l4v" event={"ID":"fee60708-57c1-4b5d-97b6-e5ea3ed089c1","Type":"ContainerStarted","Data":"3541ef459e5b93d6946ce5f0fe42b1a2296599d5fd742956bf2bafd28025153b"} Jul 12 21:53:38.206276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:38.206244 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mtcjn" event={"ID":"4841a61b-e23a-4fab-ab80-6705746aaf9a","Type":"ContainerStarted","Data":"a355af96daddd690316490d853babfa617813b78550ebc16d40cd91f3c9bb97c"} Jul 12 21:53:39.212770 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.212734 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mtcjn" event={"ID":"4841a61b-e23a-4fab-ab80-6705746aaf9a","Type":"ContainerStarted","Data":"c1177a4abc579050303bcb4d5572400c19a7eef1cdbae5a4d47d2b38f3055a41"} Jul 12 21:53:39.213279 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.212798 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mtcjn" event={"ID":"4841a61b-e23a-4fab-ab80-6705746aaf9a","Type":"ContainerStarted","Data":"df458fcec55e1b3865a751b9ad13035aca6f95fc0d54c0aff440ecf73a5197d7"} Jul 12 21:53:39.231484 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.231378 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mtcjn" podStartSLOduration=66.256768267 podStartE2EDuration="1m7.231354012s" totalImagesPullingTime="974.585745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:39.230819426 +0000 UTC m=+67.928307006" watchObservedRunningTime="2026-07-12 21:53:39.231354012 +0000 UTC m=+67.928841594" Jul 12 21:53:39.717721 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.717676 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m"] Jul 12 21:53:39.738643 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.738609 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m"] Jul 12 21:53:39.738876 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.738725 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:39.741491 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.741294 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 12 21:53:39.741491 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.741381 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ptcb6\"" Jul 12 21:53:39.807740 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.807695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j7x6m\" (UID: \"2bfb785b-992f-42fa-8641-22121b69acd2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:39.909117 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:39.909064 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j7x6m\" (UID: \"2bfb785b-992f-42fa-8641-22121b69acd2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:39.909378 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:39.909234 2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 12 21:53:39.909378 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:53:39.909303 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates podName:2bfb785b-992f-42fa-8641-22121b69acd2 nodeName:}" failed. No retries permitted until 2026-07-12 21:53:40.409280858 +0000 UTC m=+69.106768420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-j7x6m" (UID: "2bfb785b-992f-42fa-8641-22121b69acd2") : secret "prometheus-operator-admission-webhook-tls" not found Jul 12 21:53:40.414139 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:40.414039 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j7x6m\" (UID: \"2bfb785b-992f-42fa-8641-22121b69acd2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:40.419222 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:40.419149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2bfb785b-992f-42fa-8641-22121b69acd2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-j7x6m\" (UID: \"2bfb785b-992f-42fa-8641-22121b69acd2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:40.652239 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:40.651915 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:40.846103 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:40.846067 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m"] Jul 12 21:53:41.221961 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:41.221918 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" event={"ID":"2bfb785b-992f-42fa-8641-22121b69acd2","Type":"ContainerStarted","Data":"771eac1cd27420788ffe15758316c51744ec5ecaed22d13511c9485f3a2a6fb0"} Jul 12 21:53:41.828238 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:41.828124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:41.831327 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:41.831261 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ac3cb4e-fb16-4f25-a377-bb22b67ad865-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4lxsf\" (UID: \"6ac3cb4e-fb16-4f25-a377-bb22b67ad865\") " pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:42.107430 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:42.107348 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwml9\"" Jul 12 21:53:42.114909 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:42.114877 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4lxsf" Jul 12 21:53:43.970756 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:43.970665 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4lxsf"] Jul 12 21:53:43.975641 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:43.975601 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac3cb4e_fb16_4f25_a377_bb22b67ad865.slice/crio-7aa93f7b7c05f29a4eb63db676ef53816f1affe5c6f8867efd2408776aa012a6 WatchSource:0}: Error finding container 7aa93f7b7c05f29a4eb63db676ef53816f1affe5c6f8867efd2408776aa012a6: Status 404 returned error can't find the container with id 7aa93f7b7c05f29a4eb63db676ef53816f1affe5c6f8867efd2408776aa012a6 Jul 12 21:53:44.239021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:44.238972 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-st2cg" event={"ID":"7b709736-c734-4679-b2d4-2388270e8d67","Type":"ContainerStarted","Data":"c26360c2d29dbfd638a36562173c966147c61f30cf9dc6b90b44d23f1b1a3346"} Jul 12 21:53:44.239251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:44.239125 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:53:44.241162 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:44.241120 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c7l4v" event={"ID":"fee60708-57c1-4b5d-97b6-e5ea3ed089c1","Type":"ContainerStarted","Data":"4a43cb391d6910323c74b36c1f61c10896443c4f6581ab668d4c1190fdbe3e1f"} Jul 12 21:53:44.242738 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:44.242575 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lxsf" event={"ID":"6ac3cb4e-fb16-4f25-a377-bb22b67ad865","Type":"ContainerStarted","Data":"7aa93f7b7c05f29a4eb63db676ef53816f1affe5c6f8867efd2408776aa012a6"} Jul 12 21:53:44.259148 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:44.259083 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-st2cg" podStartSLOduration=66.531530048 podStartE2EDuration="1m12.259056474s" totalImagesPullingTime="5.727526426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:44.257516979 +0000 UTC m=+72.955004560" watchObservedRunningTime="2026-07-12 21:53:44.259056474 +0000 UTC m=+72.956544055" Jul 12 21:53:45.251076 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:45.250577 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" event={"ID":"2bfb785b-992f-42fa-8641-22121b69acd2","Type":"ContainerStarted","Data":"c52432b516e956a361f6161eb852af70ff456300ab84516360b849f9c6955c7d"} Jul 12 21:53:45.251076 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:45.251031 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:45.258001 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:45.257968 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" Jul 12 21:53:45.270669 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:45.270593 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c7l4v" podStartSLOduration=67.219078631 podStartE2EDuration="1m13.270567465s" totalImagesPullingTime="6.051488834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:44.275007409 +0000 UTC m=+72.972494990" watchObservedRunningTime="2026-07-12 21:53:45.270567465 +0000 UTC m=+73.968055046" Jul 12 21:53:45.290224 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:45.290091 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-j7x6m" podStartSLOduration=2.597651012 podStartE2EDuration="6.290058799s" totalImagesPullingTime="3.692407787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:45.270877116 +0000 UTC m=+73.968364703" watchObservedRunningTime="2026-07-12 21:53:45.290058799 +0000 UTC m=+73.987546380" Jul 12 21:53:47.069068 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.069033 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:53:47.073062 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.073029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.076139 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.076110 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 21:53:47.078800 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.078771 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 21:53:47.079884 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.079858 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 21:53:47.080010 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.079899 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gtz7b\"" Jul 12 21:53:47.080174 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.080157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 21:53:47.080406 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.080389 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 21:53:47.084059 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.084038 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 21:53:47.086644 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.086617 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:53:47.175650 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175618 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.175823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175660 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.175823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175697 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.175823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.175823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.175823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.176016 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.175838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zv2q\" (UniqueName: \"kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276418 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276437 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276510 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zv2q\" (UniqueName: \"kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.276623 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.276609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.277189 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.277157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.277494 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.277468 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.277545 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.277504 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.277631 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.277610 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.279013 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.278991 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.279328 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.279309 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.287097 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.287061 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zv2q\" (UniqueName: \"kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q\") pod \"console-5c6c6f8787-d8fgs\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:47.385891 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:47.385859 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:53:51.192535 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.192483 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-96m6h"] Jul 12 21:53:51.195263 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.195233 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.200627 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.200401 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 21:53:51.201751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.201329 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 21:53:51.201751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.201548 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 21:53:51.201751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.201623 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f6vqh\"" Jul 12 21:53:51.201751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.201673 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 21:53:51.202521 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.202497 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 21:53:51.202521 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.202515 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 21:53:51.308581 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-wtmp\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308581 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308472 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-metrics-client-ca\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308581 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308508 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-tls\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308581 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-sys\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308581 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308992 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308646 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6z8v\" (UniqueName: \"kubernetes.io/projected/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-kube-api-access-d6z8v\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308992 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308725 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-root\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308992 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-textfile\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.308992 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.308805 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.410432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.410391 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-textfile\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.410432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.410445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.410686 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.410480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-wtmp\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411311 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-textfile\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411437 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-metrics-client-ca\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411437 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411406 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-tls\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411547 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-sys\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411547 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411547 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6z8v\" (UniqueName: \"kubernetes.io/projected/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-kube-api-access-d6z8v\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411681 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-root\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411681 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411653 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-root\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.411813 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.411790 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-wtmp\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.412642 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.412455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-sys\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.413021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.412970 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-metrics-client-ca\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.413341 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.413312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.418252 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.418193 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-tls\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.419397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.419342 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.425370 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.425343 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6z8v\" (UniqueName: \"kubernetes.io/projected/31fce4f4-ead7-401d-b2f7-2e8c867c76ee-kube-api-access-d6z8v\") pod \"node-exporter-96m6h\" (UID: \"31fce4f4-ead7-401d-b2f7-2e8c867c76ee\") " pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:51.509326 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:51.509226 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96m6h" Jul 12 21:53:52.418431 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.418277 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:53:52.421165 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.421138 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.424224 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.424178 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 12 21:53:52.435986 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.435948 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 12 21:53:52.436161 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.436007 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 12 21:53:52.436161 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.436063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 12 21:53:52.436161 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.436109 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 12 21:53:52.436349 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.436171 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 12 21:53:52.437353 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.437320 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 12 21:53:52.437353 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.437342 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 12 21:53:52.437567 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.437394 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-srbhx\"" Jul 12 21:53:52.437567 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.437539 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 12 21:53:52.456911 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.456872 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:53:52.522402 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522402 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522430 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9b6h\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522583 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522601 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522621 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522965 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522673 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522965 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522965 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.522965 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.522829 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624187 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624245 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624270 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9b6h\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624397 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624406 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624433 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624513 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624558 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.624747 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.624647 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.625757 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.625725 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.625887 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.625834 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.628327 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.628272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.628460 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.628422 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.628524 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.628461 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.628976 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.628941 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.629064 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.628985 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.629692 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.629376 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.629692 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.629500 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.629692 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.629666 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.630593 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.630569 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.643358 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.643321 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9b6h\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h\") pod \"alertmanager-main-0\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:52.733443 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:52.733346 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:53:54.319313 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.319276 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7"] Jul 12 21:53:54.334841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.334807 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.346542 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.346513 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 12 21:53:54.346965 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.346943 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 12 21:53:54.347360 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.347340 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 12 21:53:54.347459 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.347419 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 12 21:53:54.347823 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.347798 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-f264j\"" Jul 12 21:53:54.347927 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.347865 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eiip8pero0g8l\"" Jul 12 21:53:54.348037 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.348014 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 12 21:53:54.360098 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.360064 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7"] Jul 12 21:53:54.441005 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.440956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441174 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441061 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zssc9\" (UniqueName: \"kubernetes.io/projected/a95b2530-6d67-45f1-8a00-7a84e8924bf1-kube-api-access-zssc9\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441174 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-grpc-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441308 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441222 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95b2530-6d67-45f1-8a00-7a84e8924bf1-metrics-client-ca\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441308 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441269 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441308 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441300 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441433 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441323 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.441433 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.441350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542433 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542394 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542508 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542546 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542633 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542615 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zssc9\" (UniqueName: \"kubernetes.io/projected/a95b2530-6d67-45f1-8a00-7a84e8924bf1-kube-api-access-zssc9\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-grpc-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.542907 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.542688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95b2530-6d67-45f1-8a00-7a84e8924bf1-metrics-client-ca\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.543589 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.543400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a95b2530-6d67-45f1-8a00-7a84e8924bf1-metrics-client-ca\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.546184 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.546155 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.546555 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.546504 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.547193 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.547164 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.547698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.547609 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-grpc-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.547698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.547675 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-tls\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.548243 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.548185 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a95b2530-6d67-45f1-8a00-7a84e8924bf1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.552979 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.552924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zssc9\" (UniqueName: \"kubernetes.io/projected/a95b2530-6d67-45f1-8a00-7a84e8924bf1-kube-api-access-zssc9\") pod \"thanos-querier-85cd65ffbf-zmfj7\" (UID: \"a95b2530-6d67-45f1-8a00-7a84e8924bf1\") " pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:54.646467 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:54.646420 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:53:55.588611 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:55.588572 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fce4f4_ead7_401d_b2f7_2e8c867c76ee.slice/crio-6589f8f937dded527288879d0ae68ce9bb6f998ec46c3d926175437a170b40f0 WatchSource:0}: Error finding container 6589f8f937dded527288879d0ae68ce9bb6f998ec46c3d926175437a170b40f0: Status 404 returned error can't find the container with id 6589f8f937dded527288879d0ae68ce9bb6f998ec46c3d926175437a170b40f0 Jul 12 21:53:55.773980 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:55.773868 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:53:55.774900 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:55.774842 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee602877_88cf_4830_9095_c71c7a334c85.slice/crio-5c39cc540a8451e20c884908370e27191d336207b6d21f4d34fb4129a35e467e WatchSource:0}: Error finding container 5c39cc540a8451e20c884908370e27191d336207b6d21f4d34fb4129a35e467e: Status 404 returned error can't find the container with id 5c39cc540a8451e20c884908370e27191d336207b6d21f4d34fb4129a35e467e Jul 12 21:53:55.795024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:55.790706 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:53:55.816890 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:55.816863 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7"] Jul 12 21:53:55.826346 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:53:55.826315 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda95b2530_6d67_45f1_8a00_7a84e8924bf1.slice/crio-3b9f20d28c7fab81cf93932bf7761052cb5edbf1a669a63bf23bc62fb09b0dd5 WatchSource:0}: Error finding container 3b9f20d28c7fab81cf93932bf7761052cb5edbf1a669a63bf23bc62fb09b0dd5: Status 404 returned error can't find the container with id 3b9f20d28c7fab81cf93932bf7761052cb5edbf1a669a63bf23bc62fb09b0dd5 Jul 12 21:53:55.958625 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:55.958540 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:53:56.301582 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.301470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lxsf" event={"ID":"6ac3cb4e-fb16-4f25-a377-bb22b67ad865","Type":"ContainerStarted","Data":"d85404b1080b827d53ec382f5f23de09efcbfbd47386c69d0599c573173f68fd"} Jul 12 21:53:56.305667 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.305561 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6c6f8787-d8fgs" event={"ID":"380f9a57-9e80-4719-b699-21d477a212c2","Type":"ContainerStarted","Data":"d9ee639fd9e7dce59308a8a2bfaa66080adcec21b58227dff7fb999526f9f022"} Jul 12 21:53:56.309551 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.309437 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96m6h" event={"ID":"31fce4f4-ead7-401d-b2f7-2e8c867c76ee","Type":"ContainerStarted","Data":"6589f8f937dded527288879d0ae68ce9bb6f998ec46c3d926175437a170b40f0"} Jul 12 21:53:56.312983 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.312787 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-d8vlr" event={"ID":"3a74edf0-42f2-4bad-b978-6f91498803cd","Type":"ContainerStarted","Data":"acdbfecb9aa132e3fd9bfd319f39a77558bf84201aa5c0dbaab21bd0b8abbf67"} Jul 12 21:53:56.313617 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.313551 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:56.316063 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.315962 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"5c39cc540a8451e20c884908370e27191d336207b6d21f4d34fb4129a35e467e"} Jul 12 21:53:56.318490 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.317718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"3b9f20d28c7fab81cf93932bf7761052cb5edbf1a669a63bf23bc62fb09b0dd5"} Jul 12 21:53:56.318490 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.317954 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-d8vlr" Jul 12 21:53:56.337640 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:56.337300 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-d8vlr" podStartSLOduration=1.965854032 podStartE2EDuration="21.33719855s" totalImagesPullingTime="19.371344518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:53:56.336407259 +0000 UTC m=+85.033894839" watchObservedRunningTime="2026-07-12 21:53:56.33719855 +0000 UTC m=+85.034686131" Jul 12 21:53:58.214601 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:58.214222 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fd87957d-4h4ld" Jul 12 21:53:58.332300 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:58.332195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96m6h" event={"ID":"31fce4f4-ead7-401d-b2f7-2e8c867c76ee","Type":"ContainerStarted","Data":"b5aaad120ab8198772f10002b2e118c16cb68deeaa88a3fcafba1cac6d45b664"} Jul 12 21:53:58.336517 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:58.336420 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c"} Jul 12 21:53:58.339611 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:53:58.339470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lxsf" event={"ID":"6ac3cb4e-fb16-4f25-a377-bb22b67ad865","Type":"ContainerStarted","Data":"4e0436f17ae9c5597cc398ea24a9e4501c769dafc8cd1982746c2b4d4aeda84c"} Jul 12 21:54:00.353241 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:00.352730 2575 generic.go:356] "Generic (PLEG): container finished" podID="31fce4f4-ead7-401d-b2f7-2e8c867c76ee" containerID="b5aaad120ab8198772f10002b2e118c16cb68deeaa88a3fcafba1cac6d45b664" exitCode=0 Jul 12 21:54:00.353241 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:00.352838 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96m6h" event={"ID":"31fce4f4-ead7-401d-b2f7-2e8c867c76ee","Type":"ContainerDied","Data":"b5aaad120ab8198772f10002b2e118c16cb68deeaa88a3fcafba1cac6d45b664"} Jul 12 21:54:00.356463 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:00.356400 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c" exitCode=0 Jul 12 21:54:00.356463 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:00.356469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c"} Jul 12 21:54:01.368001 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:01.367286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6c6f8787-d8fgs" event={"ID":"380f9a57-9e80-4719-b699-21d477a212c2","Type":"ContainerStarted","Data":"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0"} Jul 12 21:54:01.373356 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:01.373315 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96m6h" event={"ID":"31fce4f4-ead7-401d-b2f7-2e8c867c76ee","Type":"ContainerStarted","Data":"ecde9050ba15f83f81b9431063c9068684c78da9a0c1cccaf55a57fdab3b716a"} Jul 12 21:54:01.381802 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:01.381703 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"724eba059a6cbe2a596353a110a5408fa59f8085f41ec87d09ae8691a4376439"} Jul 12 21:54:02.391462 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:02.391414 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"3d6dd765220a139acec8dabfb35b54adf6f3101a969b92cbd6d06d186428e6ca"} Jul 12 21:54:02.391932 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:02.391511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"09dc91d03b07523d4e5eff81ece4575e15d15acf213cc90943f47da7959d3803"} Jul 12 21:54:02.395119 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:02.395081 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96m6h" event={"ID":"31fce4f4-ead7-401d-b2f7-2e8c867c76ee","Type":"ContainerStarted","Data":"dd41d2ec5176cbd82319b6898bdbe58655a5b2b9032e8deb1cf689f015f5ba5b"} Jul 12 21:54:02.421419 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:02.421351 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c6c6f8787-d8fgs" podStartSLOduration=10.159898795 podStartE2EDuration="15.421329528s" totalImagesPullingTime="5.261430733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:54:01.4008555 +0000 UTC m=+90.098343085" watchObservedRunningTime="2026-07-12 21:54:02.421329528 +0000 UTC m=+91.118817108" Jul 12 21:54:02.421634 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:02.421487 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-96m6h" podStartSLOduration=8.768366898 podStartE2EDuration="11.421478717s" totalImagesPullingTime="2.525708498s" totalInitContainerRuntime="127.403321ms" isStatefulPod=true podCreationTimestamp="2026-07-12 21:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:54:02.420363727 +0000 UTC m=+91.117851332" watchObservedRunningTime="2026-07-12 21:54:02.421478717 +0000 UTC m=+91.118966298" Jul 12 21:54:05.415317 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f"} Jul 12 21:54:05.415722 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99"} Jul 12 21:54:05.415722 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415432 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324"} Jul 12 21:54:05.415722 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415463 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96"} Jul 12 21:54:05.415722 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415486 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2"} Jul 12 21:54:05.415722 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.415509 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerStarted","Data":"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902"} Jul 12 21:54:05.418255 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.418226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"4ac39ccf9e18b55052f0b8bc6e3a219551bdf6e27d99e4f3740669874b4b56bc"} Jul 12 21:54:05.418474 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.418292 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"0c378628554a919ac15a067199356cca776cab0e3bd7f3ad33d5986b5dcd5af6"} Jul 12 21:54:05.418474 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.418320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" event={"ID":"a95b2530-6d67-45f1-8a00-7a84e8924bf1","Type":"ContainerStarted","Data":"2536cf55c4b28a44902ce4ff41903d174f6673536a4c0513d9e80e28af2a11e5"} Jul 12 21:54:05.418474 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.418355 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:54:05.420262 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.420236 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4lxsf" event={"ID":"6ac3cb4e-fb16-4f25-a377-bb22b67ad865","Type":"ContainerStarted","Data":"5994aee695f2dd443463b3f01a7bc1677c22170d573585ae95f0d3cfaad7116e"} Jul 12 21:54:05.447290 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.447199 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.151352777 podStartE2EDuration="13.447183887s" totalImagesPullingTime="6.263638653s" totalInitContainerRuntime="32.192457ms" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:54:05.44501176 +0000 UTC m=+94.142499340" watchObservedRunningTime="2026-07-12 21:54:05.447183887 +0000 UTC m=+94.144671466" Jul 12 21:54:05.467505 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.467446 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" podStartSLOduration=3.114361738 podStartE2EDuration="11.467429998s" totalImagesPullingTime="8.35306826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:53:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:54:05.467250823 +0000 UTC m=+94.164738407" watchObservedRunningTime="2026-07-12 21:54:05.467429998 +0000 UTC m=+94.164917578" Jul 12 21:54:05.487929 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:05.487865 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4lxsf" podStartSLOduration=36.439153648 podStartE2EDuration="56.487843137s" totalImagesPullingTime="20.048689489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 21:53:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:54:05.486938309 +0000 UTC m=+94.184425890" watchObservedRunningTime="2026-07-12 21:54:05.487843137 +0000 UTC m=+94.185330718" Jul 12 21:54:06.431266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:06.431127 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-85cd65ffbf-zmfj7" Jul 12 21:54:07.386558 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:07.386505 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:54:15.253786 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:15.253751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-st2cg" Jul 12 21:54:26.406811 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.406759 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c6c6f8787-d8fgs" podUID="380f9a57-9e80-4719-b699-21d477a212c2" containerName="console" containerID="cri-o://cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0" gracePeriod=15 Jul 12 21:54:26.655086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.655052 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:54:26.729867 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.729755 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.729867 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.729807 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.729867 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.729830 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.730146 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.729984 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.730146 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730028 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zv2q\" (UniqueName: \"kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.730146 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730069 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.730146 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730113 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert\") pod \"380f9a57-9e80-4719-b699-21d477a212c2\" (UID: \"380f9a57-9e80-4719-b699-21d477a212c2\") " Jul 12 21:54:26.730388 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730369 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:54:26.730446 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730304 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config" (OuterVolumeSpecName: "console-config") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:54:26.730589 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730493 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:54:26.730682 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.730625 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:54:26.732101 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.732073 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:54:26.732221 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.732190 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:54:26.732282 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.732176 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q" (OuterVolumeSpecName: "kube-api-access-6zv2q") pod "380f9a57-9e80-4719-b699-21d477a212c2" (UID: "380f9a57-9e80-4719-b699-21d477a212c2"). InnerVolumeSpecName "kube-api-access-6zv2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:54:26.831548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831503 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831543 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zv2q\" (UniqueName: \"kubernetes.io/projected/380f9a57-9e80-4719-b699-21d477a212c2-kube-api-access-6zv2q\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831556 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-trusted-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831566 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-oauth-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831807 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831575 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-serving-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831807 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831583 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/380f9a57-9e80-4719-b699-21d477a212c2-console-oauth-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:26.831807 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:26.831591 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/380f9a57-9e80-4719-b699-21d477a212c2-console-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:54:27.506049 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.506009 2575 generic.go:356] "Generic (PLEG): container finished" podID="380f9a57-9e80-4719-b699-21d477a212c2" containerID="cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0" exitCode=0 Jul 12 21:54:27.506503 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.506056 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c6c6f8787-d8fgs" Jul 12 21:54:27.506503 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.506074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6c6f8787-d8fgs" event={"ID":"380f9a57-9e80-4719-b699-21d477a212c2","Type":"ContainerDied","Data":"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0"} Jul 12 21:54:27.506503 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.506124 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c6c6f8787-d8fgs" event={"ID":"380f9a57-9e80-4719-b699-21d477a212c2","Type":"ContainerDied","Data":"d9ee639fd9e7dce59308a8a2bfaa66080adcec21b58227dff7fb999526f9f022"} Jul 12 21:54:27.506503 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.506147 2575 scope.go:117] "RemoveContainer" containerID="cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0" Jul 12 21:54:27.515008 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.514987 2575 scope.go:117] "RemoveContainer" containerID="cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0" Jul 12 21:54:27.515362 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:54:27.515337 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0\": container with ID starting with cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0 not found: ID does not exist" containerID="cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0" Jul 12 21:54:27.515432 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.515373 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0"} err="failed to get container status \"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0\": rpc error: code = NotFound desc = could not find container \"cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0\": container with ID starting with cd6aa18a43dc79cea6681e5784b22de9922ae93c142cb423f5a9a7deca4f13e0 not found: ID does not exist" Jul 12 21:54:27.527868 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.527842 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:54:27.534305 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.534281 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c6c6f8787-d8fgs"] Jul 12 21:54:27.786235 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:54:27.786130 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="380f9a57-9e80-4719-b699-21d477a212c2" path="/var/lib/kubelet/pods/380f9a57-9e80-4719-b699-21d477a212c2/volumes" Jul 12 21:55:11.627724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.627687 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.627965 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="alertmanager" containerID="cri-o://f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902" gracePeriod=120 Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.628049 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-metric" containerID="cri-o://5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99" gracePeriod=120 Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.628069 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="config-reloader" containerID="cri-o://20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2" gracePeriod=120 Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.628049 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-web" containerID="cri-o://3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96" gracePeriod=120 Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.628108 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy" containerID="cri-o://426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324" gracePeriod=120 Jul 12 21:55:11.628249 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:11.628127 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="prom-label-proxy" containerID="cri-o://bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f" gracePeriod=120 Jul 12 21:55:12.676737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676699 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f" exitCode=0 Jul 12 21:55:12.676737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676731 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99" exitCode=0 Jul 12 21:55:12.676737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676740 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324" exitCode=0 Jul 12 21:55:12.676737 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676747 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2" exitCode=0 Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676754 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902" exitCode=0 Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f"} Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676864 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99"} Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324"} Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2"} Jul 12 21:55:12.677266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.676939 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902"} Jul 12 21:55:12.871421 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:12.871394 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.000562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000461 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000507 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000531 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000561 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000604 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000625 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000648 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000672 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.000908 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.000850 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 21:55:13.001177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001011 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.001177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001047 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.001177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001093 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.001177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001122 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.001414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001177 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9b6h\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h\") pod \"ee602877-88cf-4830-9095-c71c7a334c85\" (UID: \"ee602877-88cf-4830-9095-c71c7a334c85\") " Jul 12 21:55:13.001469 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.001434 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-main-db\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.002389 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.002355 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:55:13.002591 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.002566 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:55:13.003668 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.003634 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:55:13.004098 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.004062 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.004332 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.004305 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.004456 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.004335 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume" (OuterVolumeSpecName: "config-volume") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.004528 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.004495 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.004679 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.004663 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h" (OuterVolumeSpecName: "kube-api-access-m9b6h") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "kube-api-access-m9b6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:55:13.005083 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.005057 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out" (OuterVolumeSpecName: "config-out") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 21:55:13.005155 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.005094 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.008234 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.008189 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.015767 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.015728 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config" (OuterVolumeSpecName: "web-config") pod "ee602877-88cf-4830-9095-c71c7a334c85" (UID: "ee602877-88cf-4830-9095-c71c7a334c85"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:55:13.102122 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102058 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102122 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102112 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-config-volume\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102122 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102123 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-tls-assets\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102122 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102133 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-main-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102122 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102144 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-web-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102152 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102162 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102170 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee602877-88cf-4830-9095-c71c7a334c85-metrics-client-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102179 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-cluster-tls-config\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102187 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9b6h\" (UniqueName: \"kubernetes.io/projected/ee602877-88cf-4830-9095-c71c7a334c85-kube-api-access-m9b6h\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102195 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee602877-88cf-4830-9095-c71c7a334c85-config-out\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.102471 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.102203 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee602877-88cf-4830-9095-c71c7a334c85-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:55:13.683564 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.683528 2575 generic.go:356] "Generic (PLEG): container finished" podID="ee602877-88cf-4830-9095-c71c7a334c85" containerID="3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96" exitCode=0 Jul 12 21:55:13.683974 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.683606 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96"} Jul 12 21:55:13.683974 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.683619 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.683974 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.683683 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee602877-88cf-4830-9095-c71c7a334c85","Type":"ContainerDied","Data":"5c39cc540a8451e20c884908370e27191d336207b6d21f4d34fb4129a35e467e"} Jul 12 21:55:13.683974 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.683719 2575 scope.go:117] "RemoveContainer" containerID="bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f" Jul 12 21:55:13.691814 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.691787 2575 scope.go:117] "RemoveContainer" containerID="5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99" Jul 12 21:55:13.699608 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.699581 2575 scope.go:117] "RemoveContainer" containerID="426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324" Jul 12 21:55:13.707039 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.707019 2575 scope.go:117] "RemoveContainer" containerID="3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96" Jul 12 21:55:13.710283 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.710246 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:13.714585 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.714556 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:13.718481 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.718457 2575 scope.go:117] "RemoveContainer" containerID="20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2" Jul 12 21:55:13.726624 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.726603 2575 scope.go:117] "RemoveContainer" containerID="f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902" Jul 12 21:55:13.734706 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.734684 2575 scope.go:117] "RemoveContainer" containerID="631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c" Jul 12 21:55:13.741385 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741356 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:13.741691 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741674 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741695 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741710 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="init-config-reloader" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741718 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="init-config-reloader" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741732 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380f9a57-9e80-4719-b699-21d477a212c2" containerName="console" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741742 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="380f9a57-9e80-4719-b699-21d477a212c2" containerName="console" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741753 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="config-reloader" Jul 12 21:55:13.741772 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741761 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="config-reloader" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741782 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-metric" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741792 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-metric" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741803 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="prom-label-proxy" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741811 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="prom-label-proxy" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741821 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-web" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741829 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-web" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741842 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="alertmanager" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741850 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="alertmanager" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741918 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-web" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741930 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="380f9a57-9e80-4719-b699-21d477a212c2" containerName="console" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741940 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="config-reloader" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741949 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741959 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="prom-label-proxy" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741968 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="alertmanager" Jul 12 21:55:13.742094 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.741977 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee602877-88cf-4830-9095-c71c7a334c85" containerName="kube-rbac-proxy-metric" Jul 12 21:55:13.742790 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.742526 2575 scope.go:117] "RemoveContainer" containerID="bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f" Jul 12 21:55:13.742902 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.742839 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f\": container with ID starting with bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f not found: ID does not exist" containerID="bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f" Jul 12 21:55:13.742902 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.742871 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f"} err="failed to get container status \"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f\": rpc error: code = NotFound desc = could not find container \"bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f\": container with ID starting with bac80958c4abdc4f5f6ad98c1d532e260807874c5265c743dd8e7fa2fbcd504f not found: ID does not exist" Jul 12 21:55:13.742902 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.742890 2575 scope.go:117] "RemoveContainer" containerID="5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99" Jul 12 21:55:13.743171 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.743151 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99\": container with ID starting with 5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99 not found: ID does not exist" containerID="5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99" Jul 12 21:55:13.743256 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743173 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99"} err="failed to get container status \"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99\": rpc error: code = NotFound desc = could not find container \"5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99\": container with ID starting with 5dd61303656d441ac03e6434271edf8b7b3f129f0a79f0629f8ef6bb6479eb99 not found: ID does not exist" Jul 12 21:55:13.743256 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743188 2575 scope.go:117] "RemoveContainer" containerID="426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324" Jul 12 21:55:13.743468 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.743448 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324\": container with ID starting with 426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324 not found: ID does not exist" containerID="426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324" Jul 12 21:55:13.743529 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743471 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324"} err="failed to get container status \"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324\": rpc error: code = NotFound desc = could not find container \"426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324\": container with ID starting with 426ed70dd41e0f153ce9cea26916ab69ffe130394ed27fa5841f6afacdadd324 not found: ID does not exist" Jul 12 21:55:13.743529 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743485 2575 scope.go:117] "RemoveContainer" containerID="3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96" Jul 12 21:55:13.743710 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.743695 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96\": container with ID starting with 3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96 not found: ID does not exist" containerID="3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96" Jul 12 21:55:13.743744 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743715 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96"} err="failed to get container status \"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96\": rpc error: code = NotFound desc = could not find container \"3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96\": container with ID starting with 3b769b9a407d16047ec69c502522b5eed0afa75bcc9720c856ef0f50cd423f96 not found: ID does not exist" Jul 12 21:55:13.743744 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743728 2575 scope.go:117] "RemoveContainer" containerID="20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2" Jul 12 21:55:13.743959 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.743944 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2\": container with ID starting with 20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2 not found: ID does not exist" containerID="20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2" Jul 12 21:55:13.744026 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743960 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2"} err="failed to get container status \"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2\": rpc error: code = NotFound desc = could not find container \"20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2\": container with ID starting with 20f4cb4ea7785ab5034061c3fe3ffc17e29f0aaf0f039e66ef4c25aae3b7f8b2 not found: ID does not exist" Jul 12 21:55:13.744026 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.743971 2575 scope.go:117] "RemoveContainer" containerID="f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902" Jul 12 21:55:13.744185 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.744169 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902\": container with ID starting with f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902 not found: ID does not exist" containerID="f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902" Jul 12 21:55:13.744250 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.744190 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902"} err="failed to get container status \"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902\": rpc error: code = NotFound desc = could not find container \"f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902\": container with ID starting with f53b6a08c4c88162e73137f8d7fef4f16d5d9f4816f00b9cfdc42f41ee28c902 not found: ID does not exist" Jul 12 21:55:13.744524 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.744202 2575 scope.go:117] "RemoveContainer" containerID="631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c" Jul 12 21:55:13.744814 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:55:13.744799 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c\": container with ID starting with 631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c not found: ID does not exist" containerID="631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c" Jul 12 21:55:13.744851 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.744818 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c"} err="failed to get container status \"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c\": rpc error: code = NotFound desc = could not find container \"631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c\": container with ID starting with 631b4b3dd9439030e6bba6e554746c1573277fc2bf86c55cf3fa525ead39451c not found: ID does not exist" Jul 12 21:55:13.745746 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.745726 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.748646 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748620 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 12 21:55:13.748761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748652 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 12 21:55:13.748761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748620 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 12 21:55:13.748761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748739 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 12 21:55:13.748761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748620 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 12 21:55:13.748938 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748660 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-srbhx\"" Jul 12 21:55:13.748997 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.748983 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 12 21:55:13.749031 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.749020 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 12 21:55:13.749077 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.749064 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 12 21:55:13.754749 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.754727 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 12 21:55:13.761349 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.761322 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:13.787521 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.787492 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee602877-88cf-4830-9095-c71c7a334c85" path="/var/lib/kubelet/pods/ee602877-88cf-4830-9095-c71c7a334c85/volumes" Jul 12 21:55:13.908966 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.908923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909152 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.908994 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909152 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909107 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909260 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909152 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909260 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909188 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909260 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909245 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909352 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htxdg\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-kube-api-access-htxdg\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909352 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-web-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909352 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909436 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909337 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909469 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909469 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909454 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-out\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:13.909526 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:13.909492 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.009922 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.009922 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.009922 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.009922 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009921 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htxdg\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-kube-api-access-htxdg\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009945 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-web-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.009987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-out\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010134 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010609 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010465 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.010906 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.010882 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.012018 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.011513 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d6ca531-8f76-44cf-b096-65c71f46e5c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.012018 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.011569 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.012018 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.011599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013193 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013303 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013539 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013412 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013539 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013502 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013539 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-web-config\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013688 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013545 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2d6ca531-8f76-44cf-b096-65c71f46e5c4-config-out\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.013869 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.013853 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.014405 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.014383 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2d6ca531-8f76-44cf-b096-65c71f46e5c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.015244 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.015222 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.018128 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.018109 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htxdg\" (UniqueName: \"kubernetes.io/projected/2d6ca531-8f76-44cf-b096-65c71f46e5c4-kube-api-access-htxdg\") pod \"alertmanager-main-0\" (UID: \"2d6ca531-8f76-44cf-b096-65c71f46e5c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.056220 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.056162 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 21:55:14.204093 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.204062 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 21:55:14.206406 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:55:14.206379 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6ca531_8f76_44cf_b096_65c71f46e5c4.slice/crio-84619f34c4ecb1e2a85aac12c180d7fd3b8dbc2efa06f96d1fee048a9fe62d82 WatchSource:0}: Error finding container 84619f34c4ecb1e2a85aac12c180d7fd3b8dbc2efa06f96d1fee048a9fe62d82: Status 404 returned error can't find the container with id 84619f34c4ecb1e2a85aac12c180d7fd3b8dbc2efa06f96d1fee048a9fe62d82 Jul 12 21:55:14.688402 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.688367 2575 generic.go:356] "Generic (PLEG): container finished" podID="2d6ca531-8f76-44cf-b096-65c71f46e5c4" containerID="72bb8522c107101feaff53233b40d7e57e5cd564a1af759051a8a0bc4f823f76" exitCode=0 Jul 12 21:55:14.688859 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.688452 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerDied","Data":"72bb8522c107101feaff53233b40d7e57e5cd564a1af759051a8a0bc4f823f76"} Jul 12 21:55:14.688859 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:14.688526 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"84619f34c4ecb1e2a85aac12c180d7fd3b8dbc2efa06f96d1fee048a9fe62d82"} Jul 12 21:55:15.659108 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.659071 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc"] Jul 12 21:55:15.662025 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.662002 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.664905 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.664878 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 12 21:55:15.664905 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.664878 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 12 21:55:15.665130 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.664939 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 12 21:55:15.665130 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.665003 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7js66\"" Jul 12 21:55:15.665422 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.665407 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 12 21:55:15.665790 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.665775 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 12 21:55:15.671522 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.671492 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 12 21:55:15.675809 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.675768 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc"] Jul 12 21:55:15.705700 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705664 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"8d6694401e978d907ab15921b1760f33da9549ad4b02afdaaccf408631662b4e"} Jul 12 21:55:15.706160 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"2bf9bd54bf806e2ab70e246d4c49219b8943c766b25fd27b9619080b865609f9"} Jul 12 21:55:15.706160 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705754 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"8558e3077c2999a29e5f9facf3b018d0768b19ae5c8a7e07b5f1b22a35c7d8b3"} Jul 12 21:55:15.706160 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705780 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"dbf13f91dffcddf4b4d9f12b9ea27b8b839812d286f812941cd2358e2644e6bc"} Jul 12 21:55:15.706160 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705808 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"06c64467776bbc8d886c650555441ee21defbb36332e2038db138238f5cc22f9"} Jul 12 21:55:15.706160 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.705832 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2d6ca531-8f76-44cf-b096-65c71f46e5c4","Type":"ContainerStarted","Data":"c3baaad6114687b5f3de9cce93621abb6ad28eaf899ce7829d29529e22814d8f"} Jul 12 21:55:15.733626 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.733566 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.713349693 podStartE2EDuration="2.7335458s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.196107ms" isStatefulPod=false podCreationTimestamp="2026-07-12 21:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:55:15.73159429 +0000 UTC m=+164.429081879" watchObservedRunningTime="2026-07-12 21:55:15.7335458 +0000 UTC m=+164.431033380" Jul 12 21:55:15.827487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827446 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-federate-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827487 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827486 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-serving-certs-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827761 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827749 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-metrics-client-ca\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827904 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7nlv\" (UniqueName: \"kubernetes.io/projected/0cf10002-9771-456a-a188-7d8177495837-kube-api-access-k7nlv\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827904 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827809 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-telemeter-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.827973 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.827945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929191 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929098 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-federate-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929191 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929384 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929384 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-serving-certs-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929384 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-metrics-client-ca\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929493 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7nlv\" (UniqueName: \"kubernetes.io/projected/0cf10002-9771-456a-a188-7d8177495837-kube-api-access-k7nlv\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929548 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-telemeter-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.929598 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.929546 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.930078 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.930044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-metrics-client-ca\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.930279 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.930249 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-serving-certs-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.930366 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.930350 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf10002-9771-456a-a188-7d8177495837-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.931841 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.931810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-federate-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.932063 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.932042 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-telemeter-client-tls\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.932183 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.932166 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.932298 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.932276 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cf10002-9771-456a-a188-7d8177495837-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.940355 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.940325 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7nlv\" (UniqueName: \"kubernetes.io/projected/0cf10002-9771-456a-a188-7d8177495837-kube-api-access-k7nlv\") pod \"telemeter-client-54fb6b9d9-g2dcc\" (UID: \"0cf10002-9771-456a-a188-7d8177495837\") " pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:15.974077 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:15.974033 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" Jul 12 21:55:16.120383 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:16.120351 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc"] Jul 12 21:55:16.124027 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:55:16.123968 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf10002_9771_456a_a188_7d8177495837.slice/crio-0a264d538ca26d73ca96fdd1b5bfcd22655e73921317a80e8d617ef3a5876318 WatchSource:0}: Error finding container 0a264d538ca26d73ca96fdd1b5bfcd22655e73921317a80e8d617ef3a5876318: Status 404 returned error can't find the container with id 0a264d538ca26d73ca96fdd1b5bfcd22655e73921317a80e8d617ef3a5876318 Jul 12 21:55:16.716306 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:16.716264 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" event={"ID":"0cf10002-9771-456a-a188-7d8177495837","Type":"ContainerStarted","Data":"0a264d538ca26d73ca96fdd1b5bfcd22655e73921317a80e8d617ef3a5876318"} Jul 12 21:55:18.726289 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:18.726247 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" event={"ID":"0cf10002-9771-456a-a188-7d8177495837","Type":"ContainerStarted","Data":"8170db7cd472bd6525429a763bcf238ff32aa56a8eb18bb15fe6938ee1e81c83"} Jul 12 21:55:18.726686 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:18.726319 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" event={"ID":"0cf10002-9771-456a-a188-7d8177495837","Type":"ContainerStarted","Data":"c93b3ec45508c560bf4ee643b260d4b7167c1c2908d7e9c7c0913ca4dcd0694b"} Jul 12 21:55:18.726686 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:18.726347 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" event={"ID":"0cf10002-9771-456a-a188-7d8177495837","Type":"ContainerStarted","Data":"905349bb38720bc08660e838367976b0a907b59d785e3f0dff71eb5b40cad8bb"} Jul 12 21:55:18.751400 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:55:18.751340 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54fb6b9d9-g2dcc" podStartSLOduration=1.801408535 podStartE2EDuration="3.751321827s" totalImagesPullingTime="1.949913292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:55:18.750450759 +0000 UTC m=+167.447938351" watchObservedRunningTime="2026-07-12 21:55:18.751321827 +0000 UTC m=+167.448809407" Jul 12 21:57:30.383989 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.383955 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t"] Jul 12 21:57:30.387497 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.387471 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.390320 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.390289 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 21:57:30.390490 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.390359 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 12 21:57:30.391465 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.391438 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 21:57:30.391634 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.391607 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 12 21:57:30.391751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.391651 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 12 21:57:30.391751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.391526 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 21:57:30.391751 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.391541 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 12 21:57:30.399991 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.399964 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t"] Jul 12 21:57:30.489995 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.489956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.489995 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.490003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.490254 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.490027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.490254 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.490098 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m8l2\" (UniqueName: \"kubernetes.io/projected/88e659a9-c669-4970-82c4-a044bc917c67-kube-api-access-8m8l2\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.490254 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.490162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.490254 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.490246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/88e659a9-c669-4970-82c4-a044bc917c67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591054 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591064 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591106 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8l2\" (UniqueName: \"kubernetes.io/projected/88e659a9-c669-4970-82c4-a044bc917c67-kube-api-access-8m8l2\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591130 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/88e659a9-c669-4970-82c4-a044bc917c67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591276 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.591980 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.591950 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/88e659a9-c669-4970-82c4-a044bc917c67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.593818 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.593781 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-ca\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.593917 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.593793 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.593917 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.593880 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.593992 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.593912 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88e659a9-c669-4970-82c4-a044bc917c67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.600719 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.600687 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8l2\" (UniqueName: \"kubernetes.io/projected/88e659a9-c669-4970-82c4-a044bc917c67-kube-api-access-8m8l2\") pod \"cluster-proxy-proxy-agent-c9f644d7-vwj8t\" (UID: \"88e659a9-c669-4970-82c4-a044bc917c67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.711257 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.711122 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" Jul 12 21:57:30.854644 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:30.854612 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t"] Jul 12 21:57:30.858162 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:57:30.858108 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88e659a9_c669_4970_82c4_a044bc917c67.slice/crio-7387d61629cc28d7a178ae21e3ff283be84f9df4778176e389173d451842aa1e WatchSource:0}: Error finding container 7387d61629cc28d7a178ae21e3ff283be84f9df4778176e389173d451842aa1e: Status 404 returned error can't find the container with id 7387d61629cc28d7a178ae21e3ff283be84f9df4778176e389173d451842aa1e Jul 12 21:57:31.229340 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:31.229299 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" event={"ID":"88e659a9-c669-4970-82c4-a044bc917c67","Type":"ContainerStarted","Data":"7387d61629cc28d7a178ae21e3ff283be84f9df4778176e389173d451842aa1e"} Jul 12 21:57:31.725782 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:31.725757 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 21:57:35.249809 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:35.249769 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" event={"ID":"88e659a9-c669-4970-82c4-a044bc917c67","Type":"ContainerStarted","Data":"a40d75f9cfad44526303d1581cab42eff3c0fe0e59a405678c5908f4837e5d70"} Jul 12 21:57:35.250248 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:35.249819 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" event={"ID":"88e659a9-c669-4970-82c4-a044bc917c67","Type":"ContainerStarted","Data":"bf7f8b187c77a2a9a7866e54475f849973970f1f798a9fa81b80affa52d8c972"} Jul 12 21:57:35.250248 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:35.249838 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" event={"ID":"88e659a9-c669-4970-82c4-a044bc917c67","Type":"ContainerStarted","Data":"224e2bff3a19edb2a2c8701b5b902faf9e9a6e9f13df5c17093200a280aa1e66"} Jul 12 21:57:35.270774 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:57:35.270723 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c9f644d7-vwj8t" podStartSLOduration=1.762749548 podStartE2EDuration="5.27070674s" totalImagesPullingTime="3.507957192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:57:35.269404556 +0000 UTC m=+303.966892138" watchObservedRunningTime="2026-07-12 21:57:35.27070674 +0000 UTC m=+303.968194374" Jul 12 21:58:16.947344 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.947303 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp"] Jul 12 21:58:16.952339 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.952318 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:16.957735 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.957702 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 21:58:16.957893 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.957843 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 21:58:16.958120 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.958105 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2p5vq\"" Jul 12 21:58:16.958189 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.958130 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 21:58:16.958417 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.958402 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 21:58:16.966143 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:16.966113 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp"] Jul 12 21:58:17.086413 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.086374 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-tmp\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.086413 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.086422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.086660 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.086441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.086660 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.086512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z557x\" (UniqueName: \"kubernetes.io/projected/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-kube-api-access-z557x\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.086660 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.086585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-manifests\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188023 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.187967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-manifests\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188023 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.188033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-tmp\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188332 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.188057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188332 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.188074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188332 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.188097 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z557x\" (UniqueName: \"kubernetes.io/projected/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-kube-api-access-z557x\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.188545 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.188522 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-manifests\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.190463 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.190435 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-tmp\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.190597 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.190574 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-webhook-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.190711 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.190696 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.205928 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.205858 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z557x\" (UniqueName: \"kubernetes.io/projected/bbf0987a-c7be-4b37-9aa7-cb729d6886ba-kube-api-access-z557x\") pod \"opendatahub-operator-controller-manager-d867dd548-rsdbp\" (UID: \"bbf0987a-c7be-4b37-9aa7-cb729d6886ba\") " pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.262045 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.261996 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:17.428251 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.428226 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp"] Jul 12 21:58:17.430679 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:17.430654 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf0987a_c7be_4b37_9aa7_cb729d6886ba.slice/crio-5789bb9b237560f641e44689dd4d75f0d489fac27a554176e5190ef30d19ea31 WatchSource:0}: Error finding container 5789bb9b237560f641e44689dd4d75f0d489fac27a554176e5190ef30d19ea31: Status 404 returned error can't find the container with id 5789bb9b237560f641e44689dd4d75f0d489fac27a554176e5190ef30d19ea31 Jul 12 21:58:17.432377 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:17.432361 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 21:58:18.418021 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:18.417978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" event={"ID":"bbf0987a-c7be-4b37-9aa7-cb729d6886ba","Type":"ContainerStarted","Data":"5789bb9b237560f641e44689dd4d75f0d489fac27a554176e5190ef30d19ea31"} Jul 12 21:58:21.432506 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:21.432470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" event={"ID":"bbf0987a-c7be-4b37-9aa7-cb729d6886ba","Type":"ContainerStarted","Data":"54dbbb9a7602863b7b3e7b738cfbf94d5a393a072f47930d453f8296260f21fa"} Jul 12 21:58:22.437691 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:22.437602 2575 generic.go:356] "Generic (PLEG): container finished" podID="bbf0987a-c7be-4b37-9aa7-cb729d6886ba" containerID="54dbbb9a7602863b7b3e7b738cfbf94d5a393a072f47930d453f8296260f21fa" exitCode=0 Jul 12 21:58:22.438085 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:22.437691 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" event={"ID":"bbf0987a-c7be-4b37-9aa7-cb729d6886ba","Type":"ContainerDied","Data":"54dbbb9a7602863b7b3e7b738cfbf94d5a393a072f47930d453f8296260f21fa"} Jul 12 21:58:23.443329 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:23.443287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" event={"ID":"bbf0987a-c7be-4b37-9aa7-cb729d6886ba","Type":"ContainerStarted","Data":"d2304b9c7245b000363b45cd148d35c3bf484ab619fb29885afeaa560afd4c29"} Jul 12 21:58:23.443779 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:23.443420 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:23.474024 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:23.473974 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" podStartSLOduration=3.173608514 podStartE2EDuration="7.473956079s" totalImagesPullingTime="3.291337317s" totalInitContainerRuntime="1.009010248s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:23.473244475 +0000 UTC m=+352.170732055" watchObservedRunningTime="2026-07-12 21:58:23.473956079 +0000 UTC m=+352.171443658" Jul 12 21:58:34.449763 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:34.449732 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d867dd548-rsdbp" Jul 12 21:58:37.154630 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.154592 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s"] Jul 12 21:58:37.163421 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.163373 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.167414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.167382 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 21:58:37.167414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.167405 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 21:58:37.167599 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.167422 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 21:58:37.167599 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.167382 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 21:58:37.191578 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.191543 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s"] Jul 12 21:58:37.260715 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.260677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xwj\" (UniqueName: \"kubernetes.io/projected/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-kube-api-access-d2xwj\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.260715 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.260722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tmp\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.260982 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.260747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.273935 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.273891 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dhns"] Jul 12 21:58:37.277407 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.277384 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.283150 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.283130 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cdnv9\"" Jul 12 21:58:37.283616 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.283596 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 12 21:58:37.295492 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.295463 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dhns"] Jul 12 21:58:37.361931 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.361900 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.362132 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.361938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmh7r\" (UniqueName: \"kubernetes.io/projected/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-kube-api-access-mmh7r\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.362132 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.361972 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.362132 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.362051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xwj\" (UniqueName: \"kubernetes.io/projected/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-kube-api-access-d2xwj\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.362132 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.362071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tmp\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.362132 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:37.362076 2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 12 21:58:37.362392 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:37.362162 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs podName:dbaf50e2-97f6-49a9-9273-a35f8f3d8c44 nodeName:}" failed. No retries permitted until 2026-07-12 21:58:37.862132628 +0000 UTC m=+366.559620205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs") pod "kube-auth-proxy-66b84d88d8-s6s7s" (UID: "dbaf50e2-97f6-49a9-9273-a35f8f3d8c44") : secret "kube-auth-proxy-tls" not found Jul 12 21:58:37.364376 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.364348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tmp\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.375154 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.375116 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xwj\" (UniqueName: \"kubernetes.io/projected/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-kube-api-access-d2xwj\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.463523 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.463423 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmh7r\" (UniqueName: \"kubernetes.io/projected/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-kube-api-access-mmh7r\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.463523 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.463484 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.463742 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:37.463584 2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 12 21:58:37.463742 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:37.463637 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert podName:5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2 nodeName:}" failed. No retries permitted until 2026-07-12 21:58:37.963623565 +0000 UTC m=+366.661111122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert") pod "odh-model-controller-7998c67dc4-2dhns" (UID: "5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2") : secret "odh-model-controller-webhook-cert" not found Jul 12 21:58:37.475597 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.475569 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmh7r\" (UniqueName: \"kubernetes.io/projected/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-kube-api-access-mmh7r\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.868062 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.867951 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.870583 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.870554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbaf50e2-97f6-49a9-9273-a35f8f3d8c44-tls-certs\") pod \"kube-auth-proxy-66b84d88d8-s6s7s\" (UID: \"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44\") " pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:37.968557 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.968518 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:37.970937 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:37.970915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2-cert\") pod \"odh-model-controller-7998c67dc4-2dhns\" (UID: \"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2\") " pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:38.074002 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.073963 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" Jul 12 21:58:38.187724 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.187685 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:38.225545 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.225519 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s"] Jul 12 21:58:38.227563 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:38.227536 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbaf50e2_97f6_49a9_9273_a35f8f3d8c44.slice/crio-a4b5bf71c67a7e6d0c7fa8eeede482221d5accfd30b0a2377e724abf798325b9 WatchSource:0}: Error finding container a4b5bf71c67a7e6d0c7fa8eeede482221d5accfd30b0a2377e724abf798325b9: Status 404 returned error can't find the container with id a4b5bf71c67a7e6d0c7fa8eeede482221d5accfd30b0a2377e724abf798325b9 Jul 12 21:58:38.336777 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.336715 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2dhns"] Jul 12 21:58:38.339383 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:38.339344 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee07fe0_ed0d_4209_a0ad_7aa22cc346e2.slice/crio-bae5f215273d1290b6da40a85b934dd13624a16a4a14c80089817aa015dfac0d WatchSource:0}: Error finding container bae5f215273d1290b6da40a85b934dd13624a16a4a14c80089817aa015dfac0d: Status 404 returned error can't find the container with id bae5f215273d1290b6da40a85b934dd13624a16a4a14c80089817aa015dfac0d Jul 12 21:58:38.453961 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.453919 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vj2hh"] Jul 12 21:58:38.459029 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.459008 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:38.461660 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.461634 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 12 21:58:38.470726 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.470692 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vj2hh"] Jul 12 21:58:38.473531 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.473506 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mksmp\" (UniqueName: \"kubernetes.io/projected/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-kube-api-access-mksmp\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:38.473639 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.473614 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:38.512542 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.512493 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" event={"ID":"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44","Type":"ContainerStarted","Data":"a4b5bf71c67a7e6d0c7fa8eeede482221d5accfd30b0a2377e724abf798325b9"} Jul 12 21:58:38.513679 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.513656 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" event={"ID":"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2","Type":"ContainerStarted","Data":"bae5f215273d1290b6da40a85b934dd13624a16a4a14c80089817aa015dfac0d"} Jul 12 21:58:38.574944 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.574903 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mksmp\" (UniqueName: \"kubernetes.io/projected/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-kube-api-access-mksmp\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:38.575130 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.575023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:38.575178 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:38.575133 2575 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 12 21:58:38.575247 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:38.575190 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs podName:b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d nodeName:}" failed. No retries permitted until 2026-07-12 21:58:39.075171373 +0000 UTC m=+367.772658931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs") pod "model-serving-api-54d89bd5fc-vj2hh" (UID: "b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d") : secret "model-serving-api-tls" not found Jul 12 21:58:38.593769 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:38.593736 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mksmp\" (UniqueName: \"kubernetes.io/projected/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-kube-api-access-mksmp\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:39.080292 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.080256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:39.084989 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.084957 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d-tls-certs\") pod \"model-serving-api-54d89bd5fc-vj2hh\" (UID: \"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d\") " pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:39.114912 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.113308 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:39.118089 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.117226 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.120330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.119969 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 21:58:39.120330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.120128 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 21:58:39.120330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.120285 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 21:58:39.120330 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.120301 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 21:58:39.121391 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.121358 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-57gzk\"" Jul 12 21:58:39.135241 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.135198 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:39.181729 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.181691 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.181906 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.181781 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.181906 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.181825 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.181906 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.181859 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.182081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.181939 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.182081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.182000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.182081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.182041 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dzdp\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.283862 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.283952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.283993 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.284031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.284081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.284119 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285086 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.284168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dzdp\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.285868 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.285414 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.287828 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.287489 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.288580 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.288553 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.290866 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.290409 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.294041 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.293709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dzdp\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.295342 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.294917 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.295342 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.295083 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token\") pod \"istiod-openshift-gateway-7574459656-99ct7\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.370168 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.370013 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:39.430525 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.430239 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:39.601042 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.590439 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vj2hh"] Jul 12 21:58:39.680509 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:39.680304 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:39.684616 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:39.684582 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b689e6a_6228_420c_ad21_1ac46cab5561.slice/crio-60297e002e7c4ce1511fec6ff3cb080d9266049245cf5f2f71fefe931797c4f1 WatchSource:0}: Error finding container 60297e002e7c4ce1511fec6ff3cb080d9266049245cf5f2f71fefe931797c4f1: Status 404 returned error can't find the container with id 60297e002e7c4ce1511fec6ff3cb080d9266049245cf5f2f71fefe931797c4f1 Jul 12 21:58:40.530082 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:40.530039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" event={"ID":"0b689e6a-6228-420c-ad21-1ac46cab5561","Type":"ContainerStarted","Data":"60297e002e7c4ce1511fec6ff3cb080d9266049245cf5f2f71fefe931797c4f1"} Jul 12 21:58:40.531602 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:40.531566 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" event={"ID":"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d","Type":"ContainerStarted","Data":"6dc97ece35e1f344102ea7bdc1051619fa163e5e5df29e14913258dfde9d8cac"} Jul 12 21:58:43.550656 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.550613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" event={"ID":"dbaf50e2-97f6-49a9-9273-a35f8f3d8c44","Type":"ContainerStarted","Data":"92c47d7b60219aca3d130bf88ccb83a3975b4b8913810592507d27b41bac5813"} Jul 12 21:58:43.552473 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.552406 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" event={"ID":"b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d","Type":"ContainerStarted","Data":"a16b0cb924ef7944042dde44f5ec48909c736cfa3e7af7d72b50e5097fd66d90"} Jul 12 21:58:43.552625 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.552510 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:58:43.554300 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.554260 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" event={"ID":"5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2","Type":"ContainerStarted","Data":"b99908e6f99b15b2e2e72875bd2efc5083d6d107a397e4535bb62b7c992f2d8e"} Jul 12 21:58:43.554418 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.554389 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:43.579904 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.579838 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66b84d88d8-s6s7s" podStartSLOduration=2.280679566 podStartE2EDuration="6.579820938s" totalImagesPullingTime="4.299141372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:43.577777124 +0000 UTC m=+372.275264685" watchObservedRunningTime="2026-07-12 21:58:43.579820938 +0000 UTC m=+372.277308518" Jul 12 21:58:43.616641 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.616586 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" podStartSLOduration=2.428366015 podStartE2EDuration="6.616563373s" totalImagesPullingTime="4.188197358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:43.615860966 +0000 UTC m=+372.313348581" watchObservedRunningTime="2026-07-12 21:58:43.616563373 +0000 UTC m=+372.314050954" Jul 12 21:58:43.658395 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.658338 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" podStartSLOduration=2.6845185259999997 podStartE2EDuration="5.658320458s" totalImagesPullingTime="2.973801932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:43.658048825 +0000 UTC m=+372.355536398" watchObservedRunningTime="2026-07-12 21:58:43.658320458 +0000 UTC m=+372.355808038" Jul 12 21:58:43.993192 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.993152 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 21:58:43.993349 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:43.993268 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 21:58:44.560752 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:44.560707 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" event={"ID":"0b689e6a-6228-420c-ad21-1ac46cab5561","Type":"ContainerStarted","Data":"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e"} Jul 12 21:58:44.561267 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:44.560793 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:44.596856 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:44.596807 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" podStartSLOduration=1.291140895 podStartE2EDuration="5.596788278s" totalImagesPullingTime="4.305647383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:44.594542348 +0000 UTC m=+373.292029929" watchObservedRunningTime="2026-07-12 21:58:44.596788278 +0000 UTC m=+373.294275858" Jul 12 21:58:45.463596 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.463554 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d"] Jul 12 21:58:45.467187 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.467156 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.470784 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.470755 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ntqqw\"" Jul 12 21:58:45.490448 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.490410 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d"] Jul 12 21:58:45.549490 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549525 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549552 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549698 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1df6-a811-41c6-8856-ae0feeeef971-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549901 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549901 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549789 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmcm4\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-kube-api-access-kmcm4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.549901 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.549808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.568578 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.568543 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:45.650474 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650432 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650664 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmcm4\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-kube-api-access-kmcm4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650664 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650785 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650785 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650695 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650785 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650719 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650785 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650761 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650991 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.650991 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.650908 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1df6-a811-41c6-8856-ae0feeeef971-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.651338 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.651150 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.651338 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.651170 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.651565 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.651547 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.651634 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.651592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.652073 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.652049 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1df6-a811-41c6-8856-ae0feeeef971-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.653178 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.653152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.653426 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.653410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.682175 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.682120 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmcm4\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-kube-api-access-kmcm4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.684216 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.684183 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1df6-a811-41c6-8856-ae0feeeef971-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44c75d\" (UID: \"ae0e1df6-a811-41c6-8856-ae0feeeef971\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.787629 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.787534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:45.944569 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:45.944504 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d"] Jul 12 21:58:45.947036 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:45.947010 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0e1df6_a811_41c6_8856_ae0feeeef971.slice/crio-ae14d227c513d8b21198cc4ba02ba0e0e025f06fab10f1cc9cc8030231c01b54 WatchSource:0}: Error finding container ae14d227c513d8b21198cc4ba02ba0e0e025f06fab10f1cc9cc8030231c01b54: Status 404 returned error can't find the container with id ae14d227c513d8b21198cc4ba02ba0e0e025f06fab10f1cc9cc8030231c01b54 Jul 12 21:58:46.572346 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:46.572308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" event={"ID":"ae0e1df6-a811-41c6-8856-ae0feeeef971","Type":"ContainerStarted","Data":"ae14d227c513d8b21198cc4ba02ba0e0e025f06fab10f1cc9cc8030231c01b54"} Jul 12 21:58:48.504518 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.504470 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7"] Jul 12 21:58:48.507756 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.507730 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.511387 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.511364 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 12 21:58:48.511511 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.511474 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-x2fqk\"" Jul 12 21:58:48.519765 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.519739 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk"] Jul 12 21:58:48.523396 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.523376 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.527479 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.527454 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 12 21:58:48.527592 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.527510 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8bcsd\"" Jul 12 21:58:48.540512 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.540480 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7"] Jul 12 21:58:48.547300 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.547268 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk"] Jul 12 21:58:48.558456 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.558406 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 21:58:48.558602 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.558494 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 21:58:48.558602 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.558524 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 21:58:48.586664 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.586617 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfwkd\" (UniqueName: \"kubernetes.io/projected/3936ebea-d272-4f7c-bb93-f2386ec51915-kube-api-access-dfwkd\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.586852 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.586716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3936ebea-d272-4f7c-bb93-f2386ec51915-cert\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.586852 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.586742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nj77\" (UniqueName: \"kubernetes.io/projected/f65507a5-3e59-4ee5-bb70-23248e52acb7-kube-api-access-6nj77\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.586852 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.586775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f65507a5-3e59-4ee5-bb70-23248e52acb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.688186 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.688143 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfwkd\" (UniqueName: \"kubernetes.io/projected/3936ebea-d272-4f7c-bb93-f2386ec51915-kube-api-access-dfwkd\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.688186 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.688187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3936ebea-d272-4f7c-bb93-f2386ec51915-cert\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.688186 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.688217 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nj77\" (UniqueName: \"kubernetes.io/projected/f65507a5-3e59-4ee5-bb70-23248e52acb7-kube-api-access-6nj77\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.688573 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.688234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f65507a5-3e59-4ee5-bb70-23248e52acb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.690927 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.690901 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f65507a5-3e59-4ee5-bb70-23248e52acb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.691051 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.690989 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3936ebea-d272-4f7c-bb93-f2386ec51915-cert\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.700356 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.700319 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfwkd\" (UniqueName: \"kubernetes.io/projected/3936ebea-d272-4f7c-bb93-f2386ec51915-kube-api-access-dfwkd\") pod \"kserve-controller-manager-d6f4b65b8-h8cc7\" (UID: \"3936ebea-d272-4f7c-bb93-f2386ec51915\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.701339 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.701311 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nj77\" (UniqueName: \"kubernetes.io/projected/f65507a5-3e59-4ee5-bb70-23248e52acb7-kube-api-access-6nj77\") pod \"llmisvc-controller-manager-777c6bd7d5-7bgfk\" (UID: \"f65507a5-3e59-4ee5-bb70-23248e52acb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:48.820088 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.819994 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:48.833875 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:48.833837 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:49.016890 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.016865 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7"] Jul 12 21:58:49.020061 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:49.020032 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3936ebea_d272_4f7c_bb93_f2386ec51915.slice/crio-63cce3caede3c5df81592eceed545dc93fd9340895928a6d3e8543c47ddb7c20 WatchSource:0}: Error finding container 63cce3caede3c5df81592eceed545dc93fd9340895928a6d3e8543c47ddb7c20: Status 404 returned error can't find the container with id 63cce3caede3c5df81592eceed545dc93fd9340895928a6d3e8543c47ddb7c20 Jul 12 21:58:49.032642 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.032617 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk"] Jul 12 21:58:49.034384 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:58:49.034342 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf65507a5_3e59_4ee5_bb70_23248e52acb7.slice/crio-1cd084d17981a318d88cc07558e3ac5cef692cb432208acdb06ba236473b1f9c WatchSource:0}: Error finding container 1cd084d17981a318d88cc07558e3ac5cef692cb432208acdb06ba236473b1f9c: Status 404 returned error can't find the container with id 1cd084d17981a318d88cc07558e3ac5cef692cb432208acdb06ba236473b1f9c Jul 12 21:58:49.331479 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.331432 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:49.331681 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.331656 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" podUID="0b689e6a-6228-420c-ad21-1ac46cab5561" containerName="discovery" containerID="cri-o://134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e" gracePeriod=30 Jul 12 21:58:49.576765 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.576736 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:49.589115 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.589076 2575 generic.go:356] "Generic (PLEG): container finished" podID="0b689e6a-6228-420c-ad21-1ac46cab5561" containerID="134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e" exitCode=0 Jul 12 21:58:49.589270 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.589137 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" Jul 12 21:58:49.589270 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.589165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" event={"ID":"0b689e6a-6228-420c-ad21-1ac46cab5561","Type":"ContainerDied","Data":"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e"} Jul 12 21:58:49.589270 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.589233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-99ct7" event={"ID":"0b689e6a-6228-420c-ad21-1ac46cab5561","Type":"ContainerDied","Data":"60297e002e7c4ce1511fec6ff3cb080d9266049245cf5f2f71fefe931797c4f1"} Jul 12 21:58:49.589270 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.589264 2575 scope.go:117] "RemoveContainer" containerID="134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e" Jul 12 21:58:49.591055 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.591006 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" event={"ID":"f65507a5-3e59-4ee5-bb70-23248e52acb7","Type":"ContainerStarted","Data":"1cd084d17981a318d88cc07558e3ac5cef692cb432208acdb06ba236473b1f9c"} Jul 12 21:58:49.592894 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.592837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" event={"ID":"3936ebea-d272-4f7c-bb93-f2386ec51915","Type":"ContainerStarted","Data":"63cce3caede3c5df81592eceed545dc93fd9340895928a6d3e8543c47ddb7c20"} Jul 12 21:58:49.595519 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.595473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" event={"ID":"ae0e1df6-a811-41c6-8856-ae0feeeef971","Type":"ContainerStarted","Data":"7d3006fbb9cf0bccd1253fa69372a00cd41335b8343b3bfb4f43a528eec7efa2"} Jul 12 21:58:49.599767 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.599730 2575 scope.go:117] "RemoveContainer" containerID="134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e" Jul 12 21:58:49.600116 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:58:49.600094 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e\": container with ID starting with 134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e not found: ID does not exist" containerID="134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e" Jul 12 21:58:49.600187 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.600128 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e"} err="failed to get container status \"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e\": rpc error: code = NotFound desc = could not find container \"134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e\": container with ID starting with 134d1791d81455dc1d8ac87ec748862b1414f0ebe262d4326e7a25b37afc344e not found: ID does not exist" Jul 12 21:58:49.653385 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.653320 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" podStartSLOduration=2.044421303 podStartE2EDuration="4.653291709s" totalImagesPullingTime="2.608870406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:49.64998985 +0000 UTC m=+378.347477480" watchObservedRunningTime="2026-07-12 21:58:49.653291709 +0000 UTC m=+378.350779290" Jul 12 21:58:49.705023 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.704985 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705230 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705057 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705230 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705133 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705230 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705168 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705383 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705312 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705383 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705369 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dzdp\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705488 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts\") pod \"0b689e6a-6228-420c-ad21-1ac46cab5561\" (UID: \"0b689e6a-6228-420c-ad21-1ac46cab5561\") " Jul 12 21:58:49.705819 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705655 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 21:58:49.705819 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.705788 2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-ca-configmap\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.708040 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.708011 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:58:49.708040 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.708030 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:58:49.708175 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.708098 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp" (OuterVolumeSpecName: "kube-api-access-8dzdp") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "kube-api-access-8dzdp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:58:49.708384 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.708357 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs" (OuterVolumeSpecName: "local-certs") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 21:58:49.708798 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.708777 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:58:49.709262 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.709234 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts" (OuterVolumeSpecName: "cacerts") pod "0b689e6a-6228-420c-ad21-1ac46cab5561" (UID: "0b689e6a-6228-420c-ad21-1ac46cab5561"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 21:58:49.788812 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.788782 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:49.795250 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.795201 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:49.806562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806524 2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b689e6a-6228-420c-ad21-1ac46cab5561-local-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.806562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806550 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.806562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806561 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dzdp\" (UniqueName: \"kubernetes.io/projected/0b689e6a-6228-420c-ad21-1ac46cab5561-kube-api-access-8dzdp\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.806562 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806571 2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-cacerts\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.806852 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806581 2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-kubeconfig\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.806852 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.806590 2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b689e6a-6228-420c-ad21-1ac46cab5561-istio-csr-dns-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:58:49.942418 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.942379 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:49.968375 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:49.968332 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-99ct7"] Jul 12 21:58:50.602380 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:50.602353 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:50.603484 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:50.603461 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44c75d" Jul 12 21:58:51.792004 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:51.791964 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b689e6a-6228-420c-ad21-1ac46cab5561" path="/var/lib/kubelet/pods/0b689e6a-6228-420c-ad21-1ac46cab5561/volumes" Jul 12 21:58:52.615554 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.615511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" event={"ID":"f65507a5-3e59-4ee5-bb70-23248e52acb7","Type":"ContainerStarted","Data":"32d9188b6f22ad02deb871c2e9d01c95d8c88afbc013624d8a6ffcd5be7043b8"} Jul 12 21:58:52.615776 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.615640 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:58:52.617128 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.617100 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" event={"ID":"3936ebea-d272-4f7c-bb93-f2386ec51915","Type":"ContainerStarted","Data":"e5c2237b568256ce4422dc90197a02ec99f1e59d73f2464d2c4d0dd35585e9af"} Jul 12 21:58:52.617266 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.617182 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:58:52.636099 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.636031 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" podStartSLOduration=1.668667533 podStartE2EDuration="4.636010652s" totalImagesPullingTime="2.967343119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:52.63494538 +0000 UTC m=+381.332432959" watchObservedRunningTime="2026-07-12 21:58:52.636010652 +0000 UTC m=+381.333498232" Jul 12 21:58:52.657862 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:52.657804 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" podStartSLOduration=1.729527369 podStartE2EDuration="4.657787527s" totalImagesPullingTime="2.928260158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:58:52.655910622 +0000 UTC m=+381.353398201" watchObservedRunningTime="2026-07-12 21:58:52.657787527 +0000 UTC m=+381.355275106" Jul 12 21:58:54.563228 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:54.563178 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2dhns" Jul 12 21:58:54.565331 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:58:54.565309 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vj2hh" Jul 12 21:59:23.625073 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:23.625040 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7bgfk" Jul 12 21:59:23.628177 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:23.628155 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h8cc7" Jul 12 21:59:37.839460 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.839417 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:37.839968 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.839947 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b689e6a-6228-420c-ad21-1ac46cab5561" containerName="discovery" Jul 12 21:59:37.840041 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.839973 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b689e6a-6228-420c-ad21-1ac46cab5561" containerName="discovery" Jul 12 21:59:37.840114 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.840101 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b689e6a-6228-420c-ad21-1ac46cab5561" containerName="discovery" Jul 12 21:59:37.844259 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.844230 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:37.847414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.847378 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 21:59:37.847414 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.847395 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 21:59:37.847646 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.847378 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mqfwv\"" Jul 12 21:59:37.856258 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.856156 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:37.940543 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:37.940498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcj5\" (UniqueName: \"kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5\") pod \"kuadrant-operator-catalog-gp4p4\" (UID: \"906b8a00-3eca-4613-91fa-6ab46fa9a7f0\") " pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:38.040982 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.040940 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcj5\" (UniqueName: \"kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5\") pod \"kuadrant-operator-catalog-gp4p4\" (UID: \"906b8a00-3eca-4613-91fa-6ab46fa9a7f0\") " pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:38.051556 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.051520 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcj5\" (UniqueName: \"kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5\") pod \"kuadrant-operator-catalog-gp4p4\" (UID: \"906b8a00-3eca-4613-91fa-6ab46fa9a7f0\") " pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:38.155818 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.155785 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:38.196012 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.195961 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:38.298932 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.298761 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:38.301916 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:59:38.301878 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906b8a00_3eca_4613_91fa_6ab46fa9a7f0.slice/crio-69a01fc1509602d7b1324ef3a0586c1adda1b41fa72e38eea258a7214af6ad68 WatchSource:0}: Error finding container 69a01fc1509602d7b1324ef3a0586c1adda1b41fa72e38eea258a7214af6ad68: Status 404 returned error can't find the container with id 69a01fc1509602d7b1324ef3a0586c1adda1b41fa72e38eea258a7214af6ad68 Jul 12 21:59:38.404421 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.404385 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prsbm"] Jul 12 21:59:38.407334 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.407284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:38.417107 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.417079 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prsbm"] Jul 12 21:59:38.545081 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.545036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwvsl\" (UniqueName: \"kubernetes.io/projected/576eae94-fea8-4dec-b1ea-5965ce3c7813-kube-api-access-pwvsl\") pod \"kuadrant-operator-catalog-prsbm\" (UID: \"576eae94-fea8-4dec-b1ea-5965ce3c7813\") " pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:38.646393 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.646349 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvsl\" (UniqueName: \"kubernetes.io/projected/576eae94-fea8-4dec-b1ea-5965ce3c7813-kube-api-access-pwvsl\") pod \"kuadrant-operator-catalog-prsbm\" (UID: \"576eae94-fea8-4dec-b1ea-5965ce3c7813\") " pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:38.655632 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.655593 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwvsl\" (UniqueName: \"kubernetes.io/projected/576eae94-fea8-4dec-b1ea-5965ce3c7813-kube-api-access-pwvsl\") pod \"kuadrant-operator-catalog-prsbm\" (UID: \"576eae94-fea8-4dec-b1ea-5965ce3c7813\") " pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:38.717946 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.717846 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:38.840151 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.840107 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" event={"ID":"906b8a00-3eca-4613-91fa-6ab46fa9a7f0","Type":"ContainerStarted","Data":"69a01fc1509602d7b1324ef3a0586c1adda1b41fa72e38eea258a7214af6ad68"} Jul 12 21:59:38.866257 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:38.866226 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-prsbm"] Jul 12 21:59:38.890554 ip-10-0-132-6 kubenswrapper[2575]: W0712 21:59:38.890503 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576eae94_fea8_4dec_b1ea_5965ce3c7813.slice/crio-7e7b9da96d1a8064d7a05c0896ba886022b0427712d7ded835d2156e0260d2d8 WatchSource:0}: Error finding container 7e7b9da96d1a8064d7a05c0896ba886022b0427712d7ded835d2156e0260d2d8: Status 404 returned error can't find the container with id 7e7b9da96d1a8064d7a05c0896ba886022b0427712d7ded835d2156e0260d2d8 Jul 12 21:59:39.847443 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:39.847396 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" event={"ID":"576eae94-fea8-4dec-b1ea-5965ce3c7813","Type":"ContainerStarted","Data":"7e7b9da96d1a8064d7a05c0896ba886022b0427712d7ded835d2156e0260d2d8"} Jul 12 21:59:40.853921 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:40.853886 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" event={"ID":"906b8a00-3eca-4613-91fa-6ab46fa9a7f0","Type":"ContainerStarted","Data":"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9"} Jul 12 21:59:40.854492 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:40.853924 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" podUID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" containerName="registry-server" containerID="cri-o://6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9" gracePeriod=2 Jul 12 21:59:40.855614 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:40.855584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" event={"ID":"576eae94-fea8-4dec-b1ea-5965ce3c7813","Type":"ContainerStarted","Data":"103bc8d4848cbc4296741815131d8102bbd07a54663327bbbe4c9488d89175b6"} Jul 12 21:59:40.872618 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:40.872568 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" podStartSLOduration=1.867436 podStartE2EDuration="3.872553161s" totalImagesPullingTime="2.005117161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:59:40.870889293 +0000 UTC m=+429.568376874" watchObservedRunningTime="2026-07-12 21:59:40.872553161 +0000 UTC m=+429.570040742" Jul 12 21:59:40.887459 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:40.887394 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" podStartSLOduration=1.470626973 podStartE2EDuration="2.887371485s" totalImagesPullingTime="1.416744512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 21:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 21:59:40.885528223 +0000 UTC m=+429.583015800" watchObservedRunningTime="2026-07-12 21:59:40.887371485 +0000 UTC m=+429.584859068" Jul 12 21:59:41.111678 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.111594 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:41.170537 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.170495 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqcj5\" (UniqueName: \"kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5\") pod \"906b8a00-3eca-4613-91fa-6ab46fa9a7f0\" (UID: \"906b8a00-3eca-4613-91fa-6ab46fa9a7f0\") " Jul 12 21:59:41.172767 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.172725 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5" (OuterVolumeSpecName: "kube-api-access-tqcj5") pod "906b8a00-3eca-4613-91fa-6ab46fa9a7f0" (UID: "906b8a00-3eca-4613-91fa-6ab46fa9a7f0"). InnerVolumeSpecName "kube-api-access-tqcj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 21:59:41.271650 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.271603 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqcj5\" (UniqueName: \"kubernetes.io/projected/906b8a00-3eca-4613-91fa-6ab46fa9a7f0-kube-api-access-tqcj5\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 21:59:41.861200 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.861160 2575 generic.go:356] "Generic (PLEG): container finished" podID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" containerID="6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9" exitCode=0 Jul 12 21:59:41.861695 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.861230 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" Jul 12 21:59:41.861695 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.861322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" event={"ID":"906b8a00-3eca-4613-91fa-6ab46fa9a7f0","Type":"ContainerDied","Data":"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9"} Jul 12 21:59:41.861695 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.861365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gp4p4" event={"ID":"906b8a00-3eca-4613-91fa-6ab46fa9a7f0","Type":"ContainerDied","Data":"69a01fc1509602d7b1324ef3a0586c1adda1b41fa72e38eea258a7214af6ad68"} Jul 12 21:59:41.861695 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.861390 2575 scope.go:117] "RemoveContainer" containerID="6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9" Jul 12 21:59:41.870891 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.870869 2575 scope.go:117] "RemoveContainer" containerID="6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9" Jul 12 21:59:41.871238 ip-10-0-132-6 kubenswrapper[2575]: E0712 21:59:41.871200 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9\": container with ID starting with 6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9 not found: ID does not exist" containerID="6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9" Jul 12 21:59:41.871289 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.871250 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9"} err="failed to get container status \"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9\": rpc error: code = NotFound desc = could not find container \"6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9\": container with ID starting with 6a40525c456766195cca9bf5b9d258ef7159726ecf26a66c4a41302d3d42c4a9 not found: ID does not exist" Jul 12 21:59:41.897043 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.897003 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:41.908193 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:41.908162 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gp4p4"] Jul 12 21:59:43.790829 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:43.790796 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" path="/var/lib/kubelet/pods/906b8a00-3eca-4613-91fa-6ab46fa9a7f0/volumes" Jul 12 21:59:48.718935 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:48.718893 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:48.719490 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:48.718965 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:48.742681 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:48.742644 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 21:59:48.918032 ip-10-0-132-6 kubenswrapper[2575]: I0712 21:59:48.918005 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-prsbm" Jul 12 22:00:07.836497 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.836409 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6"] Jul 12 22:00:07.836960 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.836790 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" containerName="registry-server" Jul 12 22:00:07.836960 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.836803 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" containerName="registry-server" Jul 12 22:00:07.836960 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.836895 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="906b8a00-3eca-4613-91fa-6ab46fa9a7f0" containerName="registry-server" Jul 12 22:00:07.840051 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.840032 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:07.842908 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.842885 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5jbd9\"" Jul 12 22:00:07.844402 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.844379 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 12 22:00:07.857990 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.857960 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6"] Jul 12 22:00:07.897902 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.897866 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gfgt\" (UniqueName: \"kubernetes.io/projected/800c07ad-1ada-4351-95dc-b0d8524f617b-kube-api-access-9gfgt\") pod \"dns-operator-controller-manager-864dd78c4c-b2gv6\" (UID: \"800c07ad-1ada-4351-95dc-b0d8524f617b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:07.998487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:07.998446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gfgt\" (UniqueName: \"kubernetes.io/projected/800c07ad-1ada-4351-95dc-b0d8524f617b-kube-api-access-9gfgt\") pod \"dns-operator-controller-manager-864dd78c4c-b2gv6\" (UID: \"800c07ad-1ada-4351-95dc-b0d8524f617b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:08.012932 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:08.012895 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gfgt\" (UniqueName: \"kubernetes.io/projected/800c07ad-1ada-4351-95dc-b0d8524f617b-kube-api-access-9gfgt\") pod \"dns-operator-controller-manager-864dd78c4c-b2gv6\" (UID: \"800c07ad-1ada-4351-95dc-b0d8524f617b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:08.150503 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:08.150457 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:08.295454 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:08.295413 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6"] Jul 12 22:00:08.299182 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:08.299155 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800c07ad_1ada_4351_95dc_b0d8524f617b.slice/crio-4082ae9ebf083a7b5dd0d6aff4f05f4d70336fe3a6b55f5c0f4c9665a3ef4832 WatchSource:0}: Error finding container 4082ae9ebf083a7b5dd0d6aff4f05f4d70336fe3a6b55f5c0f4c9665a3ef4832: Status 404 returned error can't find the container with id 4082ae9ebf083a7b5dd0d6aff4f05f4d70336fe3a6b55f5c0f4c9665a3ef4832 Jul 12 22:00:08.988585 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:08.988544 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" event={"ID":"800c07ad-1ada-4351-95dc-b0d8524f617b","Type":"ContainerStarted","Data":"4082ae9ebf083a7b5dd0d6aff4f05f4d70336fe3a6b55f5c0f4c9665a3ef4832"} Jul 12 22:00:11.001114 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.001021 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" event={"ID":"800c07ad-1ada-4351-95dc-b0d8524f617b","Type":"ContainerStarted","Data":"52611994343a661cdd136ea3cf2b0230affdde3603f0c7517c222e65c9c07c08"} Jul 12 22:00:11.001114 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.001096 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:11.020398 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.020346 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" podStartSLOduration=2.062701248 podStartE2EDuration="4.020329175s" totalImagesPullingTime="1.957627927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:11.017594719 +0000 UTC m=+459.715082300" watchObservedRunningTime="2026-07-12 22:00:11.020329175 +0000 UTC m=+459.717816755" Jul 12 22:00:11.274559 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.274472 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6"] Jul 12 22:00:11.278021 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.278000 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:11.281036 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.281017 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g98ch\"" Jul 12 22:00:11.292026 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.291995 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6"] Jul 12 22:00:11.330246 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.330186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwszm\" (UniqueName: \"kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm\") pod \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" (UID: \"259b2721-d9ea-4e7b-ad49-d3bc7da06585\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:11.430955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.430919 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwszm\" (UniqueName: \"kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm\") pod \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" (UID: \"259b2721-d9ea-4e7b-ad49-d3bc7da06585\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:11.445035 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.445000 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwszm\" (UniqueName: \"kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm\") pod \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" (UID: \"259b2721-d9ea-4e7b-ad49-d3bc7da06585\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:11.588289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.588146 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:11.732176 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:11.732149 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6"] Jul 12 22:00:11.734714 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:11.734678 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259b2721_d9ea_4e7b_ad49_d3bc7da06585.slice/crio-36a47a29b38c58b8c7165eff5758ba8599b18451435f5c2fbd6e7e59a4227f58 WatchSource:0}: Error finding container 36a47a29b38c58b8c7165eff5758ba8599b18451435f5c2fbd6e7e59a4227f58: Status 404 returned error can't find the container with id 36a47a29b38c58b8c7165eff5758ba8599b18451435f5c2fbd6e7e59a4227f58 Jul 12 22:00:12.006711 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:12.006674 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" event={"ID":"259b2721-d9ea-4e7b-ad49-d3bc7da06585","Type":"ContainerStarted","Data":"36a47a29b38c58b8c7165eff5758ba8599b18451435f5c2fbd6e7e59a4227f58"} Jul 12 22:00:14.018705 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:14.018662 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" event={"ID":"259b2721-d9ea-4e7b-ad49-d3bc7da06585","Type":"ContainerStarted","Data":"5112b29232a1bf4aacfa38496181294797c1518a3409d81ada05a2bb4100e698"} Jul 12 22:00:14.019111 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:14.018759 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:14.058687 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:14.058628 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" podStartSLOduration=1.507441739 podStartE2EDuration="3.058611482s" totalImagesPullingTime="1.551169743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:14.057625761 +0000 UTC m=+462.755113342" watchObservedRunningTime="2026-07-12 22:00:14.058611482 +0000 UTC m=+462.756099062" Jul 12 22:00:16.269404 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.269368 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml"] Jul 12 22:00:16.273092 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.273070 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.278055 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.278023 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kgzgg\"" Jul 12 22:00:16.292554 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.292521 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml"] Jul 12 22:00:16.378285 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.378240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzz7\" (UniqueName: \"kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.378285 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.378288 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.479465 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.479425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzz7\" (UniqueName: \"kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.479465 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.479469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.479825 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.479810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.489653 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.489617 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzz7\" (UniqueName: \"kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.583838 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.583749 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:16.754092 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:16.754058 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml"] Jul 12 22:00:16.757868 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:16.757840 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb4a90a_4edb_401f_a1f6_d020c3d7961d.slice/crio-c216e2b07fa837e59cb2cd2af90c534bbb3f305fbf907eac489ea1d3000fa262 WatchSource:0}: Error finding container c216e2b07fa837e59cb2cd2af90c534bbb3f305fbf907eac489ea1d3000fa262: Status 404 returned error can't find the container with id c216e2b07fa837e59cb2cd2af90c534bbb3f305fbf907eac489ea1d3000fa262 Jul 12 22:00:17.034567 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:17.034530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" event={"ID":"0fb4a90a-4edb-401f-a1f6-d020c3d7961d","Type":"ContainerStarted","Data":"c216e2b07fa837e59cb2cd2af90c534bbb3f305fbf907eac489ea1d3000fa262"} Jul 12 22:00:19.455784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.455741 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl"] Jul 12 22:00:19.459384 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.459361 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:19.462987 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.462961 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vqhx2\"" Jul 12 22:00:19.474161 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.474125 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl"] Jul 12 22:00:19.509001 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.508960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvcgb\" (UniqueName: \"kubernetes.io/projected/e97d212b-ecbc-4c60-a560-bdc7402ec138-kube-api-access-cvcgb\") pod \"authorino-operator-75cc8d5cb7-8s2sl\" (UID: \"e97d212b-ecbc-4c60-a560-bdc7402ec138\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:19.609806 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.609768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvcgb\" (UniqueName: \"kubernetes.io/projected/e97d212b-ecbc-4c60-a560-bdc7402ec138-kube-api-access-cvcgb\") pod \"authorino-operator-75cc8d5cb7-8s2sl\" (UID: \"e97d212b-ecbc-4c60-a560-bdc7402ec138\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:19.625188 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.625146 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvcgb\" (UniqueName: \"kubernetes.io/projected/e97d212b-ecbc-4c60-a560-bdc7402ec138-kube-api-access-cvcgb\") pod \"authorino-operator-75cc8d5cb7-8s2sl\" (UID: \"e97d212b-ecbc-4c60-a560-bdc7402ec138\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:19.770812 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.770710 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:19.932814 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:19.932787 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl"] Jul 12 22:00:19.935180 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:19.935153 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97d212b_ecbc_4c60_a560_bdc7402ec138.slice/crio-2f5f892adc83b92c6e42543407429f8dc850e3ba6c5b702fcb0a96a1c2aa0940 WatchSource:0}: Error finding container 2f5f892adc83b92c6e42543407429f8dc850e3ba6c5b702fcb0a96a1c2aa0940: Status 404 returned error can't find the container with id 2f5f892adc83b92c6e42543407429f8dc850e3ba6c5b702fcb0a96a1c2aa0940 Jul 12 22:00:20.051254 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:20.051133 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" event={"ID":"e97d212b-ecbc-4c60-a560-bdc7402ec138","Type":"ContainerStarted","Data":"2f5f892adc83b92c6e42543407429f8dc850e3ba6c5b702fcb0a96a1c2aa0940"} Jul 12 22:00:22.009993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:22.009959 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b2gv6" Jul 12 22:00:24.082141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.082088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" event={"ID":"e97d212b-ecbc-4c60-a560-bdc7402ec138","Type":"ContainerStarted","Data":"b528bdf62d74f1a8ac0aae50f8fa2458fe4a76275412cb95c291cf7e3cc3f3c1"} Jul 12 22:00:24.082645 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.082201 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:24.084265 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.084232 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" event={"ID":"0fb4a90a-4edb-401f-a1f6-d020c3d7961d","Type":"ContainerStarted","Data":"3a0a4d2d93228b9e7c44345102b85578fffab4e06f70efa0abdc2a6bf99f306e"} Jul 12 22:00:24.084412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.084349 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:24.106274 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.106193 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" podStartSLOduration=1.7284904970000001 podStartE2EDuration="5.106175204s" totalImagesPullingTime="3.377684707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:24.105104571 +0000 UTC m=+472.802592154" watchObservedRunningTime="2026-07-12 22:00:24.106175204 +0000 UTC m=+472.803662856" Jul 12 22:00:24.128495 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:24.128439 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" podStartSLOduration=1.561884879 podStartE2EDuration="8.128420963s" totalImagesPullingTime="6.566536084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:24.128079557 +0000 UTC m=+472.825567151" watchObservedRunningTime="2026-07-12 22:00:24.128420963 +0000 UTC m=+472.825908606" Jul 12 22:00:25.025991 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:25.025961 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:35.091346 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:35.091312 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8s2sl" Jul 12 22:00:35.091731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:35.091633 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:36.688729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.688691 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56"] Jul 12 22:00:36.693566 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.693542 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.717096 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.717058 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56"] Jul 12 22:00:36.768937 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.768898 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlgx\" (UniqueName: \"kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.768937 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.768957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.804413 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.804373 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56"] Jul 12 22:00:36.804718 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:36.804695 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-cqlgx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" podUID="809b0d41-918f-4260-bae6-263b9232f233" Jul 12 22:00:36.870443 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.870407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlgx\" (UniqueName: \"kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.870627 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.870476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.870936 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.870914 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.882625 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.882599 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlgx\" (UniqueName: \"kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx\") pod \"kuadrant-operator-controller-manager-55467d9f7-cld56\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:36.898596 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.898557 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56"] Jul 12 22:00:36.903238 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.903180 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56"] Jul 12 22:00:36.912505 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.912470 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml"] Jul 12 22:00:36.912701 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.912646 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" containerName="manager" containerID="cri-o://3a0a4d2d93228b9e7c44345102b85578fffab4e06f70efa0abdc2a6bf99f306e" gracePeriod=2 Jul 12 22:00:36.924806 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.924771 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml"] Jul 12 22:00:36.927594 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:36.927255 2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-59nml\": pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:36.940392 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.940308 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:36.940961 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.940943 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" containerName="manager" Jul 12 22:00:36.941002 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.940965 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" containerName="manager" Jul 12 22:00:36.941087 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.941076 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" containerName="manager" Jul 12 22:00:36.944530 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.944506 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:36.947349 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:36.947311 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:36.953638 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.953601 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:36.964564 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.964530 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6"] Jul 12 22:00:36.964780 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.964753 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" containerName="manager" containerID="cri-o://5112b29232a1bf4aacfa38496181294797c1518a3409d81ada05a2bb4100e698" gracePeriod=2 Jul 12 22:00:36.972047 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.972011 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:00:36.976508 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.976474 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:36.985042 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:36.985009 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6"] Jul 12 22:00:37.010375 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.010339 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:00:37.017592 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.017560 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw"] Jul 12 22:00:37.017965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.017950 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" containerName="manager" Jul 12 22:00:37.017965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.017964 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" containerName="manager" Jul 12 22:00:37.018093 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.018035 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" containerName="manager" Jul 12 22:00:37.021061 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.021031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:37.045650 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.045612 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw"] Jul 12 22:00:37.067984 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.067950 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.070498 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.070462 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.073001 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.072971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.073116 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.073032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77j55\" (UniqueName: \"kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.073116 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.073063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.073268 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.073125 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28vr\" (UniqueName: \"kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.109662 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.109617 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.112051 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.112013 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.150844 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.150756 2575 generic.go:356] "Generic (PLEG): container finished" podID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" containerID="5112b29232a1bf4aacfa38496181294797c1518a3409d81ada05a2bb4100e698" exitCode=0 Jul 12 22:00:37.153001 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.152926 2575 generic.go:356] "Generic (PLEG): container finished" podID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" containerID="3a0a4d2d93228b9e7c44345102b85578fffab4e06f70efa0abdc2a6bf99f306e" exitCode=0 Jul 12 22:00:37.153157 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.153020 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.156500 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.156431 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.159764 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.159548 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cld56\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="809b0d41-918f-4260-bae6-263b9232f233" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.162601 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.162565 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.162601 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.162578 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.165750 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.165323 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.169019 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.168988 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.171907 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.171880 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cld56\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="809b0d41-918f-4260-bae6-263b9232f233" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.174365 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174337 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.174490 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77j55\" (UniqueName: \"kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.174548 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.174600 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28vr\" (UniqueName: \"kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.174692 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174674 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6kgp\" (UniqueName: \"kubernetes.io/projected/79e7fe86-9771-4ac3-b3cf-3479f1d08589-kube-api-access-t6kgp\") pod \"limitador-operator-controller-manager-859d7b78bd-nwmrw\" (UID: \"79e7fe86-9771-4ac3-b3cf-3479f1d08589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:37.174823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174792 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.174995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.174977 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.186888 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.186832 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77j55\" (UniqueName: \"kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7px9d\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.188614 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.188584 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28vr\" (UniqueName: \"kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr\") pod \"kuadrant-operator-controller-manager-86878c769f-6qsrw\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.211815 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.211795 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.217515 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.217477 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.219285 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.219260 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.220867 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.220838 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cld56\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="809b0d41-918f-4260-bae6-263b9232f233" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.223039 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.223014 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.225497 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.225467 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-59nml\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:37.228442 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.228415 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cld56\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="809b0d41-918f-4260-bae6-263b9232f233" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:37.230860 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:37.230833 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vmzl6\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:37.275879 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.275835 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwzz7\" (UniqueName: \"kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7\") pod \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " Jul 12 22:00:37.275879 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.275882 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume\") pod \"809b0d41-918f-4260-bae6-263b9232f233\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " Jul 12 22:00:37.276130 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.275940 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume\") pod \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\" (UID: \"0fb4a90a-4edb-401f-a1f6-d020c3d7961d\") " Jul 12 22:00:37.276130 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.276029 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqlgx\" (UniqueName: \"kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx\") pod \"809b0d41-918f-4260-bae6-263b9232f233\" (UID: \"809b0d41-918f-4260-bae6-263b9232f233\") " Jul 12 22:00:37.276265 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.276151 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kgp\" (UniqueName: \"kubernetes.io/projected/79e7fe86-9771-4ac3-b3cf-3479f1d08589-kube-api-access-t6kgp\") pod \"limitador-operator-controller-manager-859d7b78bd-nwmrw\" (UID: \"79e7fe86-9771-4ac3-b3cf-3479f1d08589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:37.276265 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.276182 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "809b0d41-918f-4260-bae6-263b9232f233" (UID: "809b0d41-918f-4260-bae6-263b9232f233"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:00:37.276471 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.276439 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0fb4a90a-4edb-401f-a1f6-d020c3d7961d" (UID: "0fb4a90a-4edb-401f-a1f6-d020c3d7961d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:00:37.278278 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.278256 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7" (OuterVolumeSpecName: "kube-api-access-zwzz7") pod "0fb4a90a-4edb-401f-a1f6-d020c3d7961d" (UID: "0fb4a90a-4edb-401f-a1f6-d020c3d7961d"). InnerVolumeSpecName "kube-api-access-zwzz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:00:37.278375 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.278359 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx" (OuterVolumeSpecName: "kube-api-access-cqlgx") pod "809b0d41-918f-4260-bae6-263b9232f233" (UID: "809b0d41-918f-4260-bae6-263b9232f233"). InnerVolumeSpecName "kube-api-access-cqlgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:00:37.288841 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.288811 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6kgp\" (UniqueName: \"kubernetes.io/projected/79e7fe86-9771-4ac3-b3cf-3479f1d08589-kube-api-access-t6kgp\") pod \"limitador-operator-controller-manager-859d7b78bd-nwmrw\" (UID: \"79e7fe86-9771-4ac3-b3cf-3479f1d08589\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:37.362905 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.362861 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:37.371135 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.371094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:37.377062 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.377034 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwszm\" (UniqueName: \"kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm\") pod \"259b2721-d9ea-4e7b-ad49-d3bc7da06585\" (UID: \"259b2721-d9ea-4e7b-ad49-d3bc7da06585\") " Jul 12 22:00:37.377371 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.377353 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-extensions-socket-volume\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:37.377455 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.377375 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqlgx\" (UniqueName: \"kubernetes.io/projected/809b0d41-918f-4260-bae6-263b9232f233-kube-api-access-cqlgx\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:37.377455 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.377391 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwzz7\" (UniqueName: \"kubernetes.io/projected/0fb4a90a-4edb-401f-a1f6-d020c3d7961d-kube-api-access-zwzz7\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:37.377455 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.377405 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/809b0d41-918f-4260-bae6-263b9232f233-extensions-socket-volume\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:37.379202 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.379181 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:37.379327 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.379247 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm" (OuterVolumeSpecName: "kube-api-access-zwszm") pod "259b2721-d9ea-4e7b-ad49-d3bc7da06585" (UID: "259b2721-d9ea-4e7b-ad49-d3bc7da06585"). InnerVolumeSpecName "kube-api-access-zwszm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:00:37.479014 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.478964 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwszm\" (UniqueName: \"kubernetes.io/projected/259b2721-d9ea-4e7b-ad49-d3bc7da06585-kube-api-access-zwszm\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:37.542516 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.542430 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:37.545682 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:37.545634 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aa0db7b_4e87_4425_8b4e_59a68f64b011.slice/crio-50767e6a4a47d64f68da79d9d7bfbb4d26bfd1c21a9aed475ac32765921369ef WatchSource:0}: Error finding container 50767e6a4a47d64f68da79d9d7bfbb4d26bfd1c21a9aed475ac32765921369ef: Status 404 returned error can't find the container with id 50767e6a4a47d64f68da79d9d7bfbb4d26bfd1c21a9aed475ac32765921369ef Jul 12 22:00:37.772426 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.772391 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw"] Jul 12 22:00:37.780011 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.779986 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:00:37.782795 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:00:37.782765 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c33b7f_505d_4ecc_9f86_73f52085d068.slice/crio-8785c66c22cd87617f5575f498c63d2a2298e91eda94dec1777c1f26525e74f9 WatchSource:0}: Error finding container 8785c66c22cd87617f5575f498c63d2a2298e91eda94dec1777c1f26525e74f9: Status 404 returned error can't find the container with id 8785c66c22cd87617f5575f498c63d2a2298e91eda94dec1777c1f26525e74f9 Jul 12 22:00:37.790431 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.790405 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb4a90a-4edb-401f-a1f6-d020c3d7961d" path="/var/lib/kubelet/pods/0fb4a90a-4edb-401f-a1f6-d020c3d7961d/volumes" Jul 12 22:00:37.790774 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.790755 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="259b2721-d9ea-4e7b-ad49-d3bc7da06585" path="/var/lib/kubelet/pods/259b2721-d9ea-4e7b-ad49-d3bc7da06585/volumes" Jul 12 22:00:37.791088 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:37.791073 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="809b0d41-918f-4260-bae6-263b9232f233" path="/var/lib/kubelet/pods/809b0d41-918f-4260-bae6-263b9232f233/volumes" Jul 12 22:00:38.159721 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.159685 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" event={"ID":"87c33b7f-505d-4ecc-9f86-73f52085d068","Type":"ContainerStarted","Data":"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5"} Jul 12 22:00:38.159913 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.159740 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" event={"ID":"87c33b7f-505d-4ecc-9f86-73f52085d068","Type":"ContainerStarted","Data":"8785c66c22cd87617f5575f498c63d2a2298e91eda94dec1777c1f26525e74f9"} Jul 12 22:00:38.159913 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.159793 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:38.161436 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.161403 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" event={"ID":"7aa0db7b-4e87-4425-8b4e-59a68f64b011","Type":"ContainerStarted","Data":"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b"} Jul 12 22:00:38.161585 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.161470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:38.161585 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.161497 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" event={"ID":"7aa0db7b-4e87-4425-8b4e-59a68f64b011","Type":"ContainerStarted","Data":"50767e6a4a47d64f68da79d9d7bfbb4d26bfd1c21a9aed475ac32765921369ef"} Jul 12 22:00:38.162988 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.162963 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" event={"ID":"79e7fe86-9771-4ac3-b3cf-3479f1d08589","Type":"ContainerStarted","Data":"f09cdc210e8e3d9ffa12780347c0ca78bfcad156bd3eb06b459fb869c86dab78"} Jul 12 22:00:38.163123 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.163005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" event={"ID":"79e7fe86-9771-4ac3-b3cf-3479f1d08589","Type":"ContainerStarted","Data":"6ef791b5d0bfc4191b2897eeced2a9c1a408de199d2823c8143a24b8ee1cfbdd"} Jul 12 22:00:38.163123 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.163071 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:38.164257 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.164239 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vmzl6" Jul 12 22:00:38.164326 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.164260 2575 scope.go:117] "RemoveContainer" containerID="5112b29232a1bf4aacfa38496181294797c1518a3409d81ada05a2bb4100e698" Jul 12 22:00:38.165824 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.165806 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:38.165824 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.165816 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-59nml" Jul 12 22:00:38.173834 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.173810 2575 scope.go:117] "RemoveContainer" containerID="3a0a4d2d93228b9e7c44345102b85578fffab4e06f70efa0abdc2a6bf99f306e" Jul 12 22:00:38.191617 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.191562 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" podStartSLOduration=2.191546053 podStartE2EDuration="2.191546053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:38.190703854 +0000 UTC m=+486.888191434" watchObservedRunningTime="2026-07-12 22:00:38.191546053 +0000 UTC m=+486.889033633" Jul 12 22:00:38.221435 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.221386 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" podStartSLOduration=2.221370503 podStartE2EDuration="2.221370503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:38.219364945 +0000 UTC m=+486.916852561" watchObservedRunningTime="2026-07-12 22:00:38.221370503 +0000 UTC m=+486.918858082" Jul 12 22:00:38.221957 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:38.221929 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-cld56\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="809b0d41-918f-4260-bae6-263b9232f233" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-cld56" Jul 12 22:00:38.246535 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:38.246479 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" podStartSLOduration=2.246460395 podStartE2EDuration="2.246460395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:00:38.244408874 +0000 UTC m=+486.941896455" watchObservedRunningTime="2026-07-12 22:00:38.246460395 +0000 UTC m=+486.943947975" Jul 12 22:00:49.175056 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.175019 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:49.175552 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.175104 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:00:49.175552 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.175266 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwmrw" Jul 12 22:00:49.254928 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.254892 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:49.255116 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.255089 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" podUID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" containerName="manager" containerID="cri-o://52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b" gracePeriod=10 Jul 12 22:00:49.512864 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.512829 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:49.691665 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.691557 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume\") pod \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " Jul 12 22:00:49.691665 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.691624 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77j55\" (UniqueName: \"kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55\") pod \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\" (UID: \"7aa0db7b-4e87-4425-8b4e-59a68f64b011\") " Jul 12 22:00:49.691982 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.691956 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7aa0db7b-4e87-4425-8b4e-59a68f64b011" (UID: "7aa0db7b-4e87-4425-8b4e-59a68f64b011"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:00:49.693838 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.693809 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55" (OuterVolumeSpecName: "kube-api-access-77j55") pod "7aa0db7b-4e87-4425-8b4e-59a68f64b011" (UID: "7aa0db7b-4e87-4425-8b4e-59a68f64b011"). InnerVolumeSpecName "kube-api-access-77j55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:00:49.792467 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.792428 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aa0db7b-4e87-4425-8b4e-59a68f64b011-extensions-socket-volume\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:49.792467 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:49.792454 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77j55\" (UniqueName: \"kubernetes.io/projected/7aa0db7b-4e87-4425-8b4e-59a68f64b011-kube-api-access-77j55\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:00:50.228922 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.228883 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" containerID="52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b" exitCode=0 Jul 12 22:00:50.229377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.228923 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" event={"ID":"7aa0db7b-4e87-4425-8b4e-59a68f64b011","Type":"ContainerDied","Data":"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b"} Jul 12 22:00:50.229377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.228955 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" Jul 12 22:00:50.229377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.228985 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d" event={"ID":"7aa0db7b-4e87-4425-8b4e-59a68f64b011","Type":"ContainerDied","Data":"50767e6a4a47d64f68da79d9d7bfbb4d26bfd1c21a9aed475ac32765921369ef"} Jul 12 22:00:50.229377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.229010 2575 scope.go:117] "RemoveContainer" containerID="52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b" Jul 12 22:00:50.238474 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.238454 2575 scope.go:117] "RemoveContainer" containerID="52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b" Jul 12 22:00:50.238855 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:00:50.238827 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b\": container with ID starting with 52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b not found: ID does not exist" containerID="52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b" Jul 12 22:00:50.238970 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.238861 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b"} err="failed to get container status \"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b\": rpc error: code = NotFound desc = could not find container \"52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b\": container with ID starting with 52586210332ad60c081658f6f6302a3086cc9913b9c5c7e87cb5e4dfca32390b not found: ID does not exist" Jul 12 22:00:50.255391 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.255358 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:50.261739 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:50.261710 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7px9d"] Jul 12 22:00:51.790705 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:00:51.790672 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" path="/var/lib/kubelet/pods/7aa0db7b-4e87-4425-8b4e-59a68f64b011/volumes" Jul 12 22:01:05.588889 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.588852 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl"] Jul 12 22:01:05.589377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.589316 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" containerName="manager" Jul 12 22:01:05.589377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.589329 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" containerName="manager" Jul 12 22:01:05.589451 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.589402 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa0db7b-4e87-4425-8b4e-59a68f64b011" containerName="manager" Jul 12 22:01:05.594355 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.594329 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.597396 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.597375 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fbx2b\"" Jul 12 22:01:05.608498 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.608462 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl"] Jul 12 22:01:05.636536 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636484 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e6b489e-069f-400d-b679-655002457efa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636630 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4wh8\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-kube-api-access-r4wh8\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636665 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e6b489e-069f-400d-b679-655002457efa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636942 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636942 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636817 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.636942 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.636839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738328 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738280 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4wh8\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-kube-api-access-r4wh8\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738328 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738336 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738357 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e6b489e-069f-400d-b679-655002457efa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738554 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e6b489e-069f-400d-b679-655002457efa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738953 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738806 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738953 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738893 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.738953 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738943 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.739104 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.738994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.739355 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.739332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e6b489e-069f-400d-b679-655002457efa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.741243 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.741165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e6b489e-069f-400d-b679-655002457efa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.741376 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.741298 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e6b489e-069f-400d-b679-655002457efa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.749022 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.748981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.749185 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.749160 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4wh8\" (UniqueName: \"kubernetes.io/projected/9e6b489e-069f-400d-b679-655002457efa-kube-api-access-r4wh8\") pod \"maas-default-gateway-openshift-default-56654dcf7-twfkl\" (UID: \"9e6b489e-069f-400d-b679-655002457efa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:05.907301 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:05.907260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:06.060740 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.060709 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl"] Jul 12 22:01:06.064106 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:01:06.064067 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6b489e_069f_400d_b679_655002457efa.slice/crio-f179202e64e8cbfe2d0da38e314a9bc7e70c0061c14e1a7519112780d49c8415 WatchSource:0}: Error finding container f179202e64e8cbfe2d0da38e314a9bc7e70c0061c14e1a7519112780d49c8415: Status 404 returned error can't find the container with id f179202e64e8cbfe2d0da38e314a9bc7e70c0061c14e1a7519112780d49c8415 Jul 12 22:01:06.066194 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.066147 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:01:06.066306 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.066257 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:01:06.066306 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.066292 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:01:06.319021 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.318919 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" event={"ID":"9e6b489e-069f-400d-b679-655002457efa","Type":"ContainerStarted","Data":"4edfe8007c470736a9484fdec54f5174596badc04a361b732ebce1e2110c095e"} Jul 12 22:01:06.319021 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.318998 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" event={"ID":"9e6b489e-069f-400d-b679-655002457efa","Type":"ContainerStarted","Data":"f179202e64e8cbfe2d0da38e314a9bc7e70c0061c14e1a7519112780d49c8415"} Jul 12 22:01:06.342948 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.342872 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podStartSLOduration=1.342852958 podStartE2EDuration="1.342852958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:01:06.341699244 +0000 UTC m=+515.039186836" watchObservedRunningTime="2026-07-12 22:01:06.342852958 +0000 UTC m=+515.040340537" Jul 12 22:01:06.907554 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:06.907502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:07.913183 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:07.913147 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:08.254510 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.254418 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:08.257106 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.257082 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:08.260426 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.260402 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-58dxb\"" Jul 12 22:01:08.268720 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.268690 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:08.329521 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.329490 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:08.330614 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.330578 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:01:08.362790 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.362742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnw68\" (UniqueName: \"kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68\") pod \"authorino-58cd955675-549l8\" (UID: \"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5\") " pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:08.464113 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.464075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnw68\" (UniqueName: \"kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68\") pod \"authorino-58cd955675-549l8\" (UID: \"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5\") " pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:08.473154 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.473123 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnw68\" (UniqueName: \"kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68\") pod \"authorino-58cd955675-549l8\" (UID: \"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5\") " pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:08.568736 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.568634 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:08.720736 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:08.720702 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:08.724037 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:01:08.724005 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5cd7d85_7251_49ab_8d6d_7a32e03b83a5.slice/crio-193421ff966447235258d3458bc8e158631252a08a0313b9bba8d715e9deebaf WatchSource:0}: Error finding container 193421ff966447235258d3458bc8e158631252a08a0313b9bba8d715e9deebaf: Status 404 returned error can't find the container with id 193421ff966447235258d3458bc8e158631252a08a0313b9bba8d715e9deebaf Jul 12 22:01:09.335704 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:09.335663 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-549l8" event={"ID":"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5","Type":"ContainerStarted","Data":"193421ff966447235258d3458bc8e158631252a08a0313b9bba8d715e9deebaf"} Jul 12 22:01:12.355487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:12.355387 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-549l8" event={"ID":"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5","Type":"ContainerStarted","Data":"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537"} Jul 12 22:01:12.371933 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:12.371873 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-549l8" podStartSLOduration=1.018128498 podStartE2EDuration="4.371853739s" totalImagesPullingTime="3.353725241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:01:12.370533128 +0000 UTC m=+521.068020709" watchObservedRunningTime="2026-07-12 22:01:12.371853739 +0000 UTC m=+521.069341319" Jul 12 22:01:14.525799 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:14.525763 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:14.526230 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:14.525934 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-549l8" podUID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" containerName="authorino" containerID="cri-o://d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537" gracePeriod=30 Jul 12 22:01:14.792813 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:14.792782 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:14.927567 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:14.927531 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnw68\" (UniqueName: \"kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68\") pod \"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5\" (UID: \"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5\") " Jul 12 22:01:14.929671 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:14.929633 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68" (OuterVolumeSpecName: "kube-api-access-hnw68") pod "a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" (UID: "a5cd7d85-7251-49ab-8d6d-7a32e03b83a5"). InnerVolumeSpecName "kube-api-access-hnw68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:01:15.029071 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.028982 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnw68\" (UniqueName: \"kubernetes.io/projected/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5-kube-api-access-hnw68\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:01:15.372791 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.372696 2575 generic.go:356] "Generic (PLEG): container finished" podID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" containerID="d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537" exitCode=0 Jul 12 22:01:15.372791 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.372741 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-549l8" Jul 12 22:01:15.372791 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.372763 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-549l8" event={"ID":"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5","Type":"ContainerDied","Data":"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537"} Jul 12 22:01:15.373069 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.372803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-549l8" event={"ID":"a5cd7d85-7251-49ab-8d6d-7a32e03b83a5","Type":"ContainerDied","Data":"193421ff966447235258d3458bc8e158631252a08a0313b9bba8d715e9deebaf"} Jul 12 22:01:15.373069 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.372825 2575 scope.go:117] "RemoveContainer" containerID="d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537" Jul 12 22:01:15.382529 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.382508 2575 scope.go:117] "RemoveContainer" containerID="d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537" Jul 12 22:01:15.382882 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:01:15.382862 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537\": container with ID starting with d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537 not found: ID does not exist" containerID="d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537" Jul 12 22:01:15.382937 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.382892 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537"} err="failed to get container status \"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537\": rpc error: code = NotFound desc = could not find container \"d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537\": container with ID starting with d6eeddb90ddf64ae96f0809adbfbfc65b291aa5bb22fa6c119bbe884bdd32537 not found: ID does not exist" Jul 12 22:01:15.394784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.394751 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:15.398995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.398953 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-549l8"] Jul 12 22:01:15.787855 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:15.787818 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" path="/var/lib/kubelet/pods/a5cd7d85-7251-49ab-8d6d-7a32e03b83a5/volumes" Jul 12 22:01:44.446840 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.446802 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:01:44.447475 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.447241 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" containerName="authorino" Jul 12 22:01:44.447475 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.447252 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" containerName="authorino" Jul 12 22:01:44.447475 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.447319 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cd7d85-7251-49ab-8d6d-7a32e03b83a5" containerName="authorino" Jul 12 22:01:44.451688 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.451659 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.456080 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.456047 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 22:01:44.456365 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.456348 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 22:01:44.456429 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.456406 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9xwn\"" Jul 12 22:01:44.464595 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.464558 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:01:44.496482 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.496438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.496660 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.496523 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np6fc\" (UniqueName: \"kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.597363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.597316 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.597543 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.597418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np6fc\" (UniqueName: \"kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.597543 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:01:44.597474 2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 22:01:44.597620 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:01:44.597559 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert podName:945865f6-c4a9-4b5e-9478-463adbc4a4eb nodeName:}" failed. No retries permitted until 2026-07-12 22:01:45.097541873 +0000 UTC m=+553.795029432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert") pod "maas-controller-6d9b96bf8b-lwjcg" (UID: "945865f6-c4a9-4b5e-9478-463adbc4a4eb") : secret "maas-controller-webhook-cert" not found Jul 12 22:01:44.609611 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.609576 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np6fc\" (UniqueName: \"kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:44.630742 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.630694 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:01:44.634052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.634024 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:44.645530 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.645497 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:01:44.698063 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.697963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:44.698063 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.698021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6mf\" (UniqueName: \"kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:44.799130 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.799085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:44.799329 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.799144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6mf\" (UniqueName: \"kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:44.799329 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:01:44.799281 2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 12 22:01:44.799477 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:01:44.799361 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert podName:b182dac2-2fe8-41d4-a698-34493ba151b0 nodeName:}" failed. No retries permitted until 2026-07-12 22:01:45.299340098 +0000 UTC m=+553.996827656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert") pod "maas-controller-6df67c4d8d-mxsjx" (UID: "b182dac2-2fe8-41d4-a698-34493ba151b0") : secret "maas-controller-webhook-cert" not found Jul 12 22:01:44.809103 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:44.809071 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6mf\" (UniqueName: \"kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:45.102705 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.102604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:45.105114 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.105078 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") pod \"maas-controller-6d9b96bf8b-lwjcg\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:45.304786 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.304741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:45.307347 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.307307 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") pod \"maas-controller-6df67c4d8d-mxsjx\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:45.364014 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.363912 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:45.509404 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.509370 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:01:45.511824 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:01:45.511794 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945865f6_c4a9_4b5e_9478_463adbc4a4eb.slice/crio-45a8528d32f2f4fce5bd9d050c7ca0c88b148251b00f147177a07a2150cbf598 WatchSource:0}: Error finding container 45a8528d32f2f4fce5bd9d050c7ca0c88b148251b00f147177a07a2150cbf598: Status 404 returned error can't find the container with id 45a8528d32f2f4fce5bd9d050c7ca0c88b148251b00f147177a07a2150cbf598 Jul 12 22:01:45.536180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.536143 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" event={"ID":"945865f6-c4a9-4b5e-9478-463adbc4a4eb","Type":"ContainerStarted","Data":"45a8528d32f2f4fce5bd9d050c7ca0c88b148251b00f147177a07a2150cbf598"} Jul 12 22:01:45.546479 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.546448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:45.689244 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:45.689195 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:01:45.691879 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:01:45.691852 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb182dac2_2fe8_41d4_a698_34493ba151b0.slice/crio-00b0ed8aaf6f8313e76f704fb7d6dd1606047f0ad19ee118690a6a936da96f19 WatchSource:0}: Error finding container 00b0ed8aaf6f8313e76f704fb7d6dd1606047f0ad19ee118690a6a936da96f19: Status 404 returned error can't find the container with id 00b0ed8aaf6f8313e76f704fb7d6dd1606047f0ad19ee118690a6a936da96f19 Jul 12 22:01:46.547136 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:46.547036 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" event={"ID":"b182dac2-2fe8-41d4-a698-34493ba151b0","Type":"ContainerStarted","Data":"00b0ed8aaf6f8313e76f704fb7d6dd1606047f0ad19ee118690a6a936da96f19"} Jul 12 22:01:48.559417 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.559309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" event={"ID":"945865f6-c4a9-4b5e-9478-463adbc4a4eb","Type":"ContainerStarted","Data":"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50"} Jul 12 22:01:48.559870 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.559434 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:48.561185 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.561153 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" event={"ID":"b182dac2-2fe8-41d4-a698-34493ba151b0","Type":"ContainerStarted","Data":"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281"} Jul 12 22:01:48.561344 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.561234 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:48.582345 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.582272 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" podStartSLOduration=1.838191869 podStartE2EDuration="4.582248247s" totalImagesPullingTime="2.744056378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:01:48.579674756 +0000 UTC m=+557.277162334" watchObservedRunningTime="2026-07-12 22:01:48.582248247 +0000 UTC m=+557.279735825" Jul 12 22:01:48.599647 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:48.599590 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" podStartSLOduration=2.025717468 podStartE2EDuration="4.599569384s" totalImagesPullingTime="2.573851916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:01:48.598538647 +0000 UTC m=+557.296026244" watchObservedRunningTime="2026-07-12 22:01:48.599569384 +0000 UTC m=+557.297056984" Jul 12 22:01:59.569184 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:59.569153 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:01:59.569737 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:59.569339 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:01:59.638448 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:59.638410 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:01:59.638802 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:59.638725 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" podUID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" containerName="manager" containerID="cri-o://a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50" gracePeriod=30 Jul 12 22:01:59.895929 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:01:59.895896 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:02:00.049095 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.049051 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") pod \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " Jul 12 22:02:00.049301 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.049133 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np6fc\" (UniqueName: \"kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc\") pod \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\" (UID: \"945865f6-c4a9-4b5e-9478-463adbc4a4eb\") " Jul 12 22:02:00.051484 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.051451 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "945865f6-c4a9-4b5e-9478-463adbc4a4eb" (UID: "945865f6-c4a9-4b5e-9478-463adbc4a4eb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:02:00.051566 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.051544 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc" (OuterVolumeSpecName: "kube-api-access-np6fc") pod "945865f6-c4a9-4b5e-9478-463adbc4a4eb" (UID: "945865f6-c4a9-4b5e-9478-463adbc4a4eb"). InnerVolumeSpecName "kube-api-access-np6fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:02:00.150077 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.150041 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945865f6-c4a9-4b5e-9478-463adbc4a4eb-webhook-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:02:00.150077 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.150074 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np6fc\" (UniqueName: \"kubernetes.io/projected/945865f6-c4a9-4b5e-9478-463adbc4a4eb-kube-api-access-np6fc\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:02:00.625034 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.624942 2575 generic.go:356] "Generic (PLEG): container finished" podID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" containerID="a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50" exitCode=0 Jul 12 22:02:00.625034 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.624991 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" Jul 12 22:02:00.625515 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.625031 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" event={"ID":"945865f6-c4a9-4b5e-9478-463adbc4a4eb","Type":"ContainerDied","Data":"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50"} Jul 12 22:02:00.625515 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.625091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9b96bf8b-lwjcg" event={"ID":"945865f6-c4a9-4b5e-9478-463adbc4a4eb","Type":"ContainerDied","Data":"45a8528d32f2f4fce5bd9d050c7ca0c88b148251b00f147177a07a2150cbf598"} Jul 12 22:02:00.625515 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.625114 2575 scope.go:117] "RemoveContainer" containerID="a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50" Jul 12 22:02:00.634488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.634467 2575 scope.go:117] "RemoveContainer" containerID="a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50" Jul 12 22:02:00.634821 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:02:00.634796 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50\": container with ID starting with a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50 not found: ID does not exist" containerID="a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50" Jul 12 22:02:00.634923 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.634830 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50"} err="failed to get container status \"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50\": rpc error: code = NotFound desc = could not find container \"a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50\": container with ID starting with a4ce8cd67d611e1bdab2726e7858bda6764dc016a06f31f981527588f9074a50 not found: ID does not exist" Jul 12 22:02:00.648168 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.648124 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:02:00.652517 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:00.652478 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d9b96bf8b-lwjcg"] Jul 12 22:02:01.788307 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:01.788269 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" path="/var/lib/kubelet/pods/945865f6-c4a9-4b5e-9478-463adbc4a4eb/volumes" Jul 12 22:02:23.157432 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.157389 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v"] Jul 12 22:02:23.157979 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.157964 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" containerName="manager" Jul 12 22:02:23.158023 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.157983 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" containerName="manager" Jul 12 22:02:23.158123 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.158111 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="945865f6-c4a9-4b5e-9478-463adbc4a4eb" containerName="manager" Jul 12 22:02:23.163896 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.163861 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.167544 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.167515 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 12 22:02:23.167699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.167656 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 22:02:23.168406 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.168384 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 22:02:23.168406 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.168396 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f4rcr\"" Jul 12 22:02:23.171533 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.171505 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v"] Jul 12 22:02:23.266505 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.266466 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz68s\" (UniqueName: \"kubernetes.io/projected/48b17f97-2a30-4537-ab2e-f03b75d68972-kube-api-access-tz68s\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.266706 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.266535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.266706 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.266602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b17f97-2a30-4537-ab2e-f03b75d68972-openshift-service-ca\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.368040 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.367995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.368250 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.368068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b17f97-2a30-4537-ab2e-f03b75d68972-openshift-service-ca\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.368250 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.368140 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz68s\" (UniqueName: \"kubernetes.io/projected/48b17f97-2a30-4537-ab2e-f03b75d68972-kube-api-access-tz68s\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.368250 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:02:23.368155 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 12 22:02:23.368250 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:02:23.368247 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls podName:48b17f97-2a30-4537-ab2e-f03b75d68972 nodeName:}" failed. No retries permitted until 2026-07-12 22:02:23.868225625 +0000 UTC m=+592.565713198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls") pod "maas-api-7845b57fd5-9wk9v" (UID: "48b17f97-2a30-4537-ab2e-f03b75d68972") : secret "maas-api-serving-cert" not found Jul 12 22:02:23.368816 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.368793 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b17f97-2a30-4537-ab2e-f03b75d68972-openshift-service-ca\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.381061 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.381031 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz68s\" (UniqueName: \"kubernetes.io/projected/48b17f97-2a30-4537-ab2e-f03b75d68972-kube-api-access-tz68s\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.743724 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.743667 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-262s7"] Jul 12 22:02:23.748082 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.748047 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.751218 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.751161 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 12 22:02:23.757095 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.757066 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-262s7"] Jul 12 22:02:23.771676 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.771629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2vww\" (UniqueName: \"kubernetes.io/projected/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-kube-api-access-g2vww\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.771892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.771820 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.789663 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.789630 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-8nz58"] Jul 12 22:02:23.792247 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.792228 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.795447 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.795424 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cr2r5\"" Jul 12 22:02:23.801943 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.801913 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-8nz58"] Jul 12 22:02:23.873203 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.873160 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/411c6caf-3243-4906-bdd8-848fa8ce1b66-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.873203 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.873225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.873487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.873292 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk2g9\" (UniqueName: \"kubernetes.io/projected/411c6caf-3243-4906-bdd8-848fa8ce1b66-kube-api-access-gk2g9\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.873487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.873334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.873487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.873417 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2vww\" (UniqueName: \"kubernetes.io/projected/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-kube-api-access-g2vww\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.874249 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.874197 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.876162 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.876136 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b17f97-2a30-4537-ab2e-f03b75d68972-maas-api-tls\") pod \"maas-api-7845b57fd5-9wk9v\" (UID: \"48b17f97-2a30-4537-ab2e-f03b75d68972\") " pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:23.894302 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.894267 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2vww\" (UniqueName: \"kubernetes.io/projected/6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f-kube-api-access-g2vww\") pod \"payload-pre-processing-74584bb9f8-262s7\" (UID: \"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:23.974890 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.974855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/411c6caf-3243-4906-bdd8-848fa8ce1b66-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.974890 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.974898 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk2g9\" (UniqueName: \"kubernetes.io/projected/411c6caf-3243-4906-bdd8-848fa8ce1b66-kube-api-access-gk2g9\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.976059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.976035 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/411c6caf-3243-4906-bdd8-848fa8ce1b66-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:23.984354 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:23.984324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk2g9\" (UniqueName: \"kubernetes.io/projected/411c6caf-3243-4906-bdd8-848fa8ce1b66-kube-api-access-gk2g9\") pod \"payload-processing-f9b8d9b9b-8nz58\" (UID: \"411c6caf-3243-4906-bdd8-848fa8ce1b66\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:24.062278 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.062134 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:24.076045 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.076017 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:24.104696 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.104601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:24.246596 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.246506 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-262s7"] Jul 12 22:02:24.249502 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:02:24.249436 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6243477e_5ad1_4f2b_8a84_d1dfbcc4bf5f.slice/crio-3ee414b7b202ec62d8b17745651fa737d36d1ae283cee9374a68a38435a9f615 WatchSource:0}: Error finding container 3ee414b7b202ec62d8b17745651fa737d36d1ae283cee9374a68a38435a9f615: Status 404 returned error can't find the container with id 3ee414b7b202ec62d8b17745651fa737d36d1ae283cee9374a68a38435a9f615 Jul 12 22:02:24.278546 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.278499 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v"] Jul 12 22:02:24.282165 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:02:24.282115 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b17f97_2a30_4537_ab2e_f03b75d68972.slice/crio-40ffbfafc5c092f3a921d0ecc265ce4245ab28858f6fcbe402d1a4f2248bef97 WatchSource:0}: Error finding container 40ffbfafc5c092f3a921d0ecc265ce4245ab28858f6fcbe402d1a4f2248bef97: Status 404 returned error can't find the container with id 40ffbfafc5c092f3a921d0ecc265ce4245ab28858f6fcbe402d1a4f2248bef97 Jul 12 22:02:24.300131 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.300100 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-8nz58"] Jul 12 22:02:24.302617 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:02:24.302585 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411c6caf_3243_4906_bdd8_848fa8ce1b66.slice/crio-7dd8319412213badcd9e52ee45f1a221eecfd6c8848dbe52202d53db0d70bcd0 WatchSource:0}: Error finding container 7dd8319412213badcd9e52ee45f1a221eecfd6c8848dbe52202d53db0d70bcd0: Status 404 returned error can't find the container with id 7dd8319412213badcd9e52ee45f1a221eecfd6c8848dbe52202d53db0d70bcd0 Jul 12 22:02:24.763919 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.763857 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" event={"ID":"411c6caf-3243-4906-bdd8-848fa8ce1b66","Type":"ContainerStarted","Data":"7dd8319412213badcd9e52ee45f1a221eecfd6c8848dbe52202d53db0d70bcd0"} Jul 12 22:02:24.765683 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.765593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" event={"ID":"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f","Type":"ContainerStarted","Data":"3ee414b7b202ec62d8b17745651fa737d36d1ae283cee9374a68a38435a9f615"} Jul 12 22:02:24.767457 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:24.767381 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" event={"ID":"48b17f97-2a30-4537-ab2e-f03b75d68972","Type":"ContainerStarted","Data":"40ffbfafc5c092f3a921d0ecc265ce4245ab28858f6fcbe402d1a4f2248bef97"} Jul 12 22:02:26.781089 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.780112 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" event={"ID":"411c6caf-3243-4906-bdd8-848fa8ce1b66","Type":"ContainerStarted","Data":"3e8f265821a972506cd2edd2f60e2342d3fe3465cae18d3e9620d10fb874a814"} Jul 12 22:02:26.781089 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.781019 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:26.785383 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.783776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" event={"ID":"6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f","Type":"ContainerStarted","Data":"a33c5c1da4d75d5602630a4ff95aa00c94fbc0f4cb94ae721681154d1ceadd4e"} Jul 12 22:02:26.785383 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.784468 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:26.791233 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.787836 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" event={"ID":"48b17f97-2a30-4537-ab2e-f03b75d68972","Type":"ContainerStarted","Data":"f580dcd8bc075043082ef7dea45da49a54bebfc66945160c0e948f4018c1ab95"} Jul 12 22:02:26.791233 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.788509 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:26.820443 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.820377 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" podStartSLOduration=1.814117684 podStartE2EDuration="3.82035103s" totalImagesPullingTime="2.006233346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:02:26.801362625 +0000 UTC m=+595.498850206" watchObservedRunningTime="2026-07-12 22:02:26.82035103 +0000 UTC m=+595.517838612" Jul 12 22:02:26.822565 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.822502 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" podStartSLOduration=1.764603175 podStartE2EDuration="3.822484954s" totalImagesPullingTime="2.057881779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:02:26.819233404 +0000 UTC m=+595.516720986" watchObservedRunningTime="2026-07-12 22:02:26.822484954 +0000 UTC m=+595.519972535" Jul 12 22:02:26.838032 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:26.837979 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" podStartSLOduration=1.810660504 podStartE2EDuration="3.837959512s" totalImagesPullingTime="2.027299008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:02:26.835869885 +0000 UTC m=+595.533357458" watchObservedRunningTime="2026-07-12 22:02:26.837959512 +0000 UTC m=+595.535447091" Jul 12 22:02:32.826949 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.826914 2575 generic.go:356] "Generic (PLEG): container finished" podID="9e6b489e-069f-400d-b679-655002457efa" containerID="4edfe8007c470736a9484fdec54f5174596badc04a361b732ebce1e2110c095e" exitCode=0 Jul 12 22:02:32.827409 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.826989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" event={"ID":"9e6b489e-069f-400d-b679-655002457efa","Type":"ContainerDied","Data":"4edfe8007c470736a9484fdec54f5174596badc04a361b732ebce1e2110c095e"} Jul 12 22:02:32.827500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.827482 2575 scope.go:117] "RemoveContainer" containerID="4edfe8007c470736a9484fdec54f5174596badc04a361b732ebce1e2110c095e" Jul 12 22:02:32.828579 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.828550 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:02:32.828697 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.828637 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:02:32.828697 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:32.828685 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:02:33.809818 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:33.809790 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7845b57fd5-9wk9v" Jul 12 22:02:33.835436 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:33.835401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" event={"ID":"9e6b489e-069f-400d-b679-655002457efa","Type":"ContainerStarted","Data":"7d80c0415f6b2bfa785cf1c626fc9811cc892b2721dca94e69cef990fa1007f3"} Jul 12 22:02:33.908509 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:33.908462 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:02:33.909492 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:33.909457 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-twfkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:02:33.909631 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:33.909522 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podUID="9e6b489e-069f-400d-b679-655002457efa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:02:34.908564 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:34.908521 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-twfkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:02:34.908982 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:34.908601 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podUID="9e6b489e-069f-400d-b679-655002457efa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:02:35.908492 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:35.908449 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-twfkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:02:35.908698 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:35.908515 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podUID="9e6b489e-069f-400d-b679-655002457efa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:02:36.908040 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:36.907994 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-twfkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:02:36.908277 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:36.908085 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podUID="9e6b489e-069f-400d-b679-655002457efa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:02:37.908580 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:37.908520 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-twfkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 12 22:02:37.908999 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:37.908602 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" podUID="9e6b489e-069f-400d-b679-655002457efa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 12 22:02:38.330410 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:38.330304 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:02:38.805688 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:38.805654 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-8nz58" Jul 12 22:02:38.805990 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:38.805957 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-262s7" Jul 12 22:02:38.912920 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:38.912882 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:02:38.913896 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:38.913874 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-twfkl" Jul 12 22:02:41.829725 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.829660 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4"] Jul 12 22:02:41.832743 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.832715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.837082 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.837051 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 22:02:41.837082 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.837051 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 12 22:02:41.837347 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.837102 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 22:02:41.837347 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.837051 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z87st\"" Jul 12 22:02:41.843378 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.843345 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4"] Jul 12 22:02:41.980628 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hkqc\" (UniqueName: \"kubernetes.io/projected/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-kube-api-access-7hkqc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.980628 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980632 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.980903 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980670 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.980903 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980728 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.980903 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980797 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:41.980903 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:41.980848 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.081906 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081815 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hkqc\" (UniqueName: \"kubernetes.io/projected/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-kube-api-access-7hkqc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.081906 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081857 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.081906 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.081906 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.082297 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.082297 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.081994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.082400 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.082344 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.082400 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.082381 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.082611 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.082591 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.084460 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.084434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.084808 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.084790 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.091140 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.091107 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hkqc\" (UniqueName: \"kubernetes.io/projected/d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0-kube-api-access-7hkqc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4\" (UID: \"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.145247 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.145199 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:42.309275 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.309241 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4"] Jul 12 22:02:42.311183 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:02:42.311155 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cddaa0_04b1_458c_a5db_2a2bd79e1fe0.slice/crio-dd557a310533b37377240606198e3da89da339323001bc44ab8f0ae16758bb29 WatchSource:0}: Error finding container dd557a310533b37377240606198e3da89da339323001bc44ab8f0ae16758bb29: Status 404 returned error can't find the container with id dd557a310533b37377240606198e3da89da339323001bc44ab8f0ae16758bb29 Jul 12 22:02:42.888043 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:42.887984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" event={"ID":"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0","Type":"ContainerStarted","Data":"dd557a310533b37377240606198e3da89da339323001bc44ab8f0ae16758bb29"} Jul 12 22:02:43.896762 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:43.896717 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" event={"ID":"d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0","Type":"ContainerStarted","Data":"d031f3f1e44df4e37fbadf400cff205d78f8421db856a530e6ad4ea4406aacd8"} Jul 12 22:02:43.916240 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:43.916149 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" podStartSLOduration=1.778116172 podStartE2EDuration="2.916131441s" totalImagesPullingTime="1.138015269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:02:43.91513781 +0000 UTC m=+612.612625390" watchObservedRunningTime="2026-07-12 22:02:43.916131441 +0000 UTC m=+612.613619022" Jul 12 22:02:52.145810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:52.145775 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:52.146264 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:52.145847 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:52.158919 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:52.158881 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:02:52.954951 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:02:52.954919 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4" Jul 12 22:03:10.649656 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.649611 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595"] Jul 12 22:03:10.655817 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.655787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.661073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.661042 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 12 22:03:10.668519 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.668467 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595"] Jul 12 22:03:10.745650 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.745829 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745661 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcrz6\" (UniqueName: \"kubernetes.io/projected/1cc13010-4c11-4f85-a231-a27361829891-kube-api-access-rcrz6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.745829 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13010-4c11-4f85-a231-a27361829891-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.745829 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745728 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.745829 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745757 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.745829 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.745779 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847352 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcrz6\" (UniqueName: \"kubernetes.io/projected/1cc13010-4c11-4f85-a231-a27361829891-kube-api-access-rcrz6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13010-4c11-4f85-a231-a27361829891-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847457 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847761 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847573 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847855 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847833 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847941 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847920 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.847995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.847954 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.849966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.849930 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc13010-4c11-4f85-a231-a27361829891-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.850232 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.850196 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13010-4c11-4f85-a231-a27361829891-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.857341 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.857312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcrz6\" (UniqueName: \"kubernetes.io/projected/1cc13010-4c11-4f85-a231-a27361829891-kube-api-access-rcrz6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595\" (UID: \"1cc13010-4c11-4f85-a231-a27361829891\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:10.968916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:10.968815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:11.124672 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:11.124643 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595"] Jul 12 22:03:11.126897 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:03:11.126866 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc13010_4c11_4f85_a231_a27361829891.slice/crio-6ad4c520532252130d0b5a122848707e5ff31274745dc4d7717d1966ecdd40f2 WatchSource:0}: Error finding container 6ad4c520532252130d0b5a122848707e5ff31274745dc4d7717d1966ecdd40f2: Status 404 returned error can't find the container with id 6ad4c520532252130d0b5a122848707e5ff31274745dc4d7717d1966ecdd40f2 Jul 12 22:03:12.048624 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:12.048583 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" event={"ID":"1cc13010-4c11-4f85-a231-a27361829891","Type":"ContainerStarted","Data":"7e877c6ed4964c813a5d51074ed27ee15d6bfde99d5993f053d736d4d819c349"} Jul 12 22:03:12.049040 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:12.048648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" event={"ID":"1cc13010-4c11-4f85-a231-a27361829891","Type":"ContainerStarted","Data":"6ad4c520532252130d0b5a122848707e5ff31274745dc4d7717d1966ecdd40f2"} Jul 12 22:03:12.068666 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:12.068612 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" podStartSLOduration=1.890205862 podStartE2EDuration="2.068593635s" totalImagesPullingTime="178.387773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:03:12.068200252 +0000 UTC m=+640.765687827" watchObservedRunningTime="2026-07-12 22:03:12.068593635 +0000 UTC m=+640.766081214" Jul 12 22:03:20.969959 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:20.969906 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:20.970551 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:20.969984 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:20.982969 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:20.982936 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:03:21.109419 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:03:21.109388 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595" Jul 12 22:05:31.676309 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.676269 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79d5d984dc-46t8v"] Jul 12 22:05:31.680102 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.680067 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.684246 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.684199 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 22:05:31.684443 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.684198 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-58dxb\"" Jul 12 22:05:31.701390 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.701352 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d5d984dc-46t8v"] Jul 12 22:05:31.797647 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.797616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbfsf\" (UniqueName: \"kubernetes.io/projected/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-kube-api-access-lbfsf\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.797841 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.797721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-tls-cert\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.898490 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.898451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-tls-cert\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.898693 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.898555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbfsf\" (UniqueName: \"kubernetes.io/projected/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-kube-api-access-lbfsf\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.901093 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.901065 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-tls-cert\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.908810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.908770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbfsf\" (UniqueName: \"kubernetes.io/projected/ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71-kube-api-access-lbfsf\") pod \"authorino-79d5d984dc-46t8v\" (UID: \"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71\") " pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:31.991425 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:31.991330 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d5d984dc-46t8v" Jul 12 22:05:32.162537 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:32.162485 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d5d984dc-46t8v"] Jul 12 22:05:32.168707 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:05:32.168642 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce8a7a5c_fcfa_4068_a5bd_af8c761c4e71.slice/crio-c07014bed45ed3ad1a554b256b816a4322bba94eb73d5b2f76e85110b3bffcc5 WatchSource:0}: Error finding container c07014bed45ed3ad1a554b256b816a4322bba94eb73d5b2f76e85110b3bffcc5: Status 404 returned error can't find the container with id c07014bed45ed3ad1a554b256b816a4322bba94eb73d5b2f76e85110b3bffcc5 Jul 12 22:05:32.170535 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:32.170508 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:05:32.806249 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:32.806126 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d5d984dc-46t8v" event={"ID":"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71","Type":"ContainerStarted","Data":"c0bc19d842d6138efd4a9fca76fdd690bc8cc2514d06d318096ebe21bc733755"} Jul 12 22:05:32.806249 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:32.806191 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d5d984dc-46t8v" event={"ID":"ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71","Type":"ContainerStarted","Data":"c07014bed45ed3ad1a554b256b816a4322bba94eb73d5b2f76e85110b3bffcc5"} Jul 12 22:05:32.841087 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:05:32.841040 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79d5d984dc-46t8v" podStartSLOduration=1.502490736 podStartE2EDuration="1.841022642s" totalImagesPullingTime="338.531906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:05:32.840991679 +0000 UTC m=+781.538479259" watchObservedRunningTime="2026-07-12 22:05:32.841022642 +0000 UTC m=+781.538510221" Jul 12 22:06:44.210498 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.210462 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:06:44.211078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.210673 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" podUID="b182dac2-2fe8-41d4-a698-34493ba151b0" containerName="manager" containerID="cri-o://e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281" gracePeriod=30 Jul 12 22:06:44.478788 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.478757 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:06:44.616699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.616655 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") pod \"b182dac2-2fe8-41d4-a698-34493ba151b0\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " Jul 12 22:06:44.616901 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.616834 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk6mf\" (UniqueName: \"kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf\") pod \"b182dac2-2fe8-41d4-a698-34493ba151b0\" (UID: \"b182dac2-2fe8-41d4-a698-34493ba151b0\") " Jul 12 22:06:44.618914 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.618877 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b182dac2-2fe8-41d4-a698-34493ba151b0" (UID: "b182dac2-2fe8-41d4-a698-34493ba151b0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:06:44.619036 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.618934 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf" (OuterVolumeSpecName: "kube-api-access-qk6mf") pod "b182dac2-2fe8-41d4-a698-34493ba151b0" (UID: "b182dac2-2fe8-41d4-a698-34493ba151b0"). InnerVolumeSpecName "kube-api-access-qk6mf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:06:44.718078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.717983 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b182dac2-2fe8-41d4-a698-34493ba151b0-webhook-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:06:44.718078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:44.718018 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk6mf\" (UniqueName: \"kubernetes.io/projected/b182dac2-2fe8-41d4-a698-34493ba151b0-kube-api-access-qk6mf\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:06:45.209629 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.209597 2575 generic.go:356] "Generic (PLEG): container finished" podID="b182dac2-2fe8-41d4-a698-34493ba151b0" containerID="e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281" exitCode=0 Jul 12 22:06:45.209804 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.209646 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" Jul 12 22:06:45.209804 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.209682 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" event={"ID":"b182dac2-2fe8-41d4-a698-34493ba151b0","Type":"ContainerDied","Data":"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281"} Jul 12 22:06:45.209804 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.209739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-mxsjx" event={"ID":"b182dac2-2fe8-41d4-a698-34493ba151b0","Type":"ContainerDied","Data":"00b0ed8aaf6f8313e76f704fb7d6dd1606047f0ad19ee118690a6a936da96f19"} Jul 12 22:06:45.209804 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.209761 2575 scope.go:117] "RemoveContainer" containerID="e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281" Jul 12 22:06:45.219944 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.219806 2575 scope.go:117] "RemoveContainer" containerID="e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281" Jul 12 22:06:45.220247 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:06:45.220148 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281\": container with ID starting with e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281 not found: ID does not exist" containerID="e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281" Jul 12 22:06:45.220247 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.220187 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281"} err="failed to get container status \"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281\": rpc error: code = NotFound desc = could not find container \"e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281\": container with ID starting with e2a5a558a999be8a62bab3b086e5d92d0cad57bf211875dbd0d9bab94ea4d281 not found: ID does not exist" Jul 12 22:06:45.230855 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.230816 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:06:45.237188 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.237155 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-mxsjx"] Jul 12 22:06:45.788593 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:45.788556 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b182dac2-2fe8-41d4-a698-34493ba151b0" path="/var/lib/kubelet/pods/b182dac2-2fe8-41d4-a698-34493ba151b0/volumes" Jul 12 22:06:46.032112 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.032075 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-hk9sv"] Jul 12 22:06:46.032541 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.032522 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b182dac2-2fe8-41d4-a698-34493ba151b0" containerName="manager" Jul 12 22:06:46.032541 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.032538 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b182dac2-2fe8-41d4-a698-34493ba151b0" containerName="manager" Jul 12 22:06:46.032735 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.032616 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b182dac2-2fe8-41d4-a698-34493ba151b0" containerName="manager" Jul 12 22:06:46.037120 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.037092 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.041384 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.041301 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 22:06:46.041384 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.041336 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9xwn\"" Jul 12 22:06:46.041384 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.041380 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 22:06:46.052030 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.051996 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-hk9sv"] Jul 12 22:06:46.130878 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.130837 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9lkb\" (UniqueName: \"kubernetes.io/projected/da1ca873-df0e-4f32-a757-dc2784b68972-kube-api-access-z9lkb\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.131058 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.130933 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da1ca873-df0e-4f32-a757-dc2784b68972-webhook-cert\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.232520 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.232479 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9lkb\" (UniqueName: \"kubernetes.io/projected/da1ca873-df0e-4f32-a757-dc2784b68972-kube-api-access-z9lkb\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.232918 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.232536 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da1ca873-df0e-4f32-a757-dc2784b68972-webhook-cert\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.235180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.235154 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/da1ca873-df0e-4f32-a757-dc2784b68972-webhook-cert\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.243575 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.243550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9lkb\" (UniqueName: \"kubernetes.io/projected/da1ca873-df0e-4f32-a757-dc2784b68972-kube-api-access-z9lkb\") pod \"maas-controller-6df67c4d8d-hk9sv\" (UID: \"da1ca873-df0e-4f32-a757-dc2784b68972\") " pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.348120 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.348018 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:46.500666 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:46.500634 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6df67c4d8d-hk9sv"] Jul 12 22:06:46.503545 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:06:46.503509 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1ca873_df0e_4f32_a757_dc2784b68972.slice/crio-541805cde6214d5bfa7be2af41acced7b39a1f6615e76e71f8ddafa1d7c3eb34 WatchSource:0}: Error finding container 541805cde6214d5bfa7be2af41acced7b39a1f6615e76e71f8ddafa1d7c3eb34: Status 404 returned error can't find the container with id 541805cde6214d5bfa7be2af41acced7b39a1f6615e76e71f8ddafa1d7c3eb34 Jul 12 22:06:47.222679 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:47.222564 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" event={"ID":"da1ca873-df0e-4f32-a757-dc2784b68972","Type":"ContainerStarted","Data":"18e2fb48fa2bc543aabefb9350ed70ca379f1ce48bd4614689fd7085b985ed52"} Jul 12 22:06:47.222679 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:47.222633 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" event={"ID":"da1ca873-df0e-4f32-a757-dc2784b68972","Type":"ContainerStarted","Data":"541805cde6214d5bfa7be2af41acced7b39a1f6615e76e71f8ddafa1d7c3eb34"} Jul 12 22:06:47.222679 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:47.222662 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:06:47.254376 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:47.254317 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" podStartSLOduration=0.841300808 podStartE2EDuration="1.25430055s" totalImagesPullingTime="412.999742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:06:47.251362827 +0000 UTC m=+855.948850417" watchObservedRunningTime="2026-07-12 22:06:47.25430055 +0000 UTC m=+855.951788136" Jul 12 22:06:58.230951 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:06:58.230920 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6df67c4d8d-hk9sv" Jul 12 22:11:40.642502 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:40.642460 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:11:40.644810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:40.642693 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" podUID="87c33b7f-505d-4ecc-9f86-73f52085d068" containerName="manager" containerID="cri-o://b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5" gracePeriod=10 Jul 12 22:11:44.416777 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.416748 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:11:44.469495 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.469404 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n28vr\" (UniqueName: \"kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr\") pod \"87c33b7f-505d-4ecc-9f86-73f52085d068\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " Jul 12 22:11:44.469495 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.469474 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume\") pod \"87c33b7f-505d-4ecc-9f86-73f52085d068\" (UID: \"87c33b7f-505d-4ecc-9f86-73f52085d068\") " Jul 12 22:11:44.469902 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.469875 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "87c33b7f-505d-4ecc-9f86-73f52085d068" (UID: "87c33b7f-505d-4ecc-9f86-73f52085d068"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:11:44.471962 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.471923 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr" (OuterVolumeSpecName: "kube-api-access-n28vr") pod "87c33b7f-505d-4ecc-9f86-73f52085d068" (UID: "87c33b7f-505d-4ecc-9f86-73f52085d068"). InnerVolumeSpecName "kube-api-access-n28vr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:11:44.571097 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.571048 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n28vr\" (UniqueName: \"kubernetes.io/projected/87c33b7f-505d-4ecc-9f86-73f52085d068-kube-api-access-n28vr\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:11:44.571097 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.571086 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/87c33b7f-505d-4ecc-9f86-73f52085d068-extensions-socket-volume\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:11:44.879729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.879622 2575 generic.go:356] "Generic (PLEG): container finished" podID="87c33b7f-505d-4ecc-9f86-73f52085d068" containerID="b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5" exitCode=0 Jul 12 22:11:44.879729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.879672 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" Jul 12 22:11:44.879729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.879712 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" event={"ID":"87c33b7f-505d-4ecc-9f86-73f52085d068","Type":"ContainerDied","Data":"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5"} Jul 12 22:11:44.879968 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.879768 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw" event={"ID":"87c33b7f-505d-4ecc-9f86-73f52085d068","Type":"ContainerDied","Data":"8785c66c22cd87617f5575f498c63d2a2298e91eda94dec1777c1f26525e74f9"} Jul 12 22:11:44.879968 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.879791 2575 scope.go:117] "RemoveContainer" containerID="b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5" Jul 12 22:11:44.889703 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.889679 2575 scope.go:117] "RemoveContainer" containerID="b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5" Jul 12 22:11:44.890014 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:11:44.889997 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5\": container with ID starting with b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5 not found: ID does not exist" containerID="b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5" Jul 12 22:11:44.890061 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.890025 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5"} err="failed to get container status \"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5\": rpc error: code = NotFound desc = could not find container \"b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5\": container with ID starting with b55a3802a8698ccd70ef53d000aa795418cfcfd4fc150f3f99d0fdaae8fad4a5 not found: ID does not exist" Jul 12 22:11:44.902309 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.902273 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:11:44.905953 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:44.905923 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6qsrw"] Jul 12 22:11:45.788432 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:11:45.788398 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87c33b7f-505d-4ecc-9f86-73f52085d068" path="/var/lib/kubelet/pods/87c33b7f-505d-4ecc-9f86-73f52085d068/volumes" Jul 12 22:12:41.777115 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.777076 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb"] Jul 12 22:12:41.777786 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.777764 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c33b7f-505d-4ecc-9f86-73f52085d068" containerName="manager" Jul 12 22:12:41.777877 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.777790 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c33b7f-505d-4ecc-9f86-73f52085d068" containerName="manager" Jul 12 22:12:41.777930 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.777900 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c33b7f-505d-4ecc-9f86-73f52085d068" containerName="manager" Jul 12 22:12:41.784905 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.784868 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:41.787925 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.787901 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kgzgg\"" Jul 12 22:12:41.797776 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.797744 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb"] Jul 12 22:12:41.920418 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.920374 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ebb49d8-1443-4205-a5cb-582db2bf94c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:41.920418 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:41.920435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44lth\" (UniqueName: \"kubernetes.io/projected/7ebb49d8-1443-4205-a5cb-582db2bf94c7-kube-api-access-44lth\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.021281 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.021233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ebb49d8-1443-4205-a5cb-582db2bf94c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.021281 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.021283 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44lth\" (UniqueName: \"kubernetes.io/projected/7ebb49d8-1443-4205-a5cb-582db2bf94c7-kube-api-access-44lth\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.021646 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.021621 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7ebb49d8-1443-4205-a5cb-582db2bf94c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.031228 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.031143 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44lth\" (UniqueName: \"kubernetes.io/projected/7ebb49d8-1443-4205-a5cb-582db2bf94c7-kube-api-access-44lth\") pod \"kuadrant-operator-controller-manager-86878c769f-ggwrb\" (UID: \"7ebb49d8-1443-4205-a5cb-582db2bf94c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.097524 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.097495 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:42.264404 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:42.264244 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb"] Jul 12 22:12:42.267833 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:12:42.267793 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebb49d8_1443_4205_a5cb_582db2bf94c7.slice/crio-635e67e1281302ce63b08fe0bbfc617d52efb970508506579102336613b841c5 WatchSource:0}: Error finding container 635e67e1281302ce63b08fe0bbfc617d52efb970508506579102336613b841c5: Status 404 returned error can't find the container with id 635e67e1281302ce63b08fe0bbfc617d52efb970508506579102336613b841c5 Jul 12 22:12:43.193057 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:43.193017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" event={"ID":"7ebb49d8-1443-4205-a5cb-582db2bf94c7","Type":"ContainerStarted","Data":"2e41e83f810d55f442acb43dc6f2b09fb5fb00b90e540ed0a7ae9511bc709dad"} Jul 12 22:12:43.193472 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:43.193068 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" event={"ID":"7ebb49d8-1443-4205-a5cb-582db2bf94c7","Type":"ContainerStarted","Data":"635e67e1281302ce63b08fe0bbfc617d52efb970508506579102336613b841c5"} Jul 12 22:12:43.193472 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:43.193141 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:12:43.216031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:43.215968 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" podStartSLOduration=2.215945859 podStartE2EDuration="2.215945859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:12:43.214861971 +0000 UTC m=+1211.912349550" watchObservedRunningTime="2026-07-12 22:12:43.215945859 +0000 UTC m=+1211.913433440" Jul 12 22:12:54.201510 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:12:54.201477 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ggwrb" Jul 12 22:19:08.625387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.625351 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:19:08.628974 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.628948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.632167 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.632140 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5249e6ec-openshift-default-dockercfg-9rqzh\"" Jul 12 22:19:08.663021 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.662981 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:19:08.670201 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670165 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670243 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670287 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670316 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670354 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670584 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670584 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670488 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lm75\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.670584 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.670526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771563 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771611 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771664 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771708 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.771757 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772060 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772060 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.771821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lm75\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772258 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.772130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772258 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.772237 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772444 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.772281 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772444 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.772405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.772594 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.772573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.774094 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.774073 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.774368 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.774353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.783922 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.783884 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lm75\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.792502 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.792470 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token\") pod \"e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:08.940504 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:08.940452 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:09.093789 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.093751 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:19:09.095800 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:19:09.095757 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0756510_b55c_41e5_b2ab_fbe1de9070de.slice/crio-aca87d03db8ffd98728fa1312e5c3f6a8e64287f6b670856b9c5c95a315c3157 WatchSource:0}: Error finding container aca87d03db8ffd98728fa1312e5c3f6a8e64287f6b670856b9c5c95a315c3157: Status 404 returned error can't find the container with id aca87d03db8ffd98728fa1312e5c3f6a8e64287f6b670856b9c5c95a315c3157 Jul 12 22:19:09.097948 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.097914 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:09.098032 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.097989 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:09.098032 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.098018 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:09.313287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.313157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerStarted","Data":"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180"} Jul 12 22:19:09.313287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.313239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerStarted","Data":"aca87d03db8ffd98728fa1312e5c3f6a8e64287f6b670856b9c5c95a315c3157"} Jul 12 22:19:09.336293 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.336241 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podStartSLOduration=1.336220556 podStartE2EDuration="1.336220556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:09.334346788 +0000 UTC m=+1598.031834369" watchObservedRunningTime="2026-07-12 22:19:09.336220556 +0000 UTC m=+1598.033708128" Jul 12 22:19:09.935823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.935782 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:19:09.939716 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.939690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:09.940788 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.940758 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:09.942332 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.942308 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5249e6ec-serving-cert\"" Jul 12 22:19:09.950721 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.950687 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:19:09.985568 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.985525 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:09.985772 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.985595 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qjjj\" (UniqueName: \"kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:09.985772 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:09.985674 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.087330 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.087284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qjjj\" (UniqueName: \"kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.087534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.087409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.087534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.087497 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.087643 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:19:10.087563 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-serving-cert: secret "maas-api-e2e-ait-5249e6ec-serving-cert" not found Jul 12 22:19:10.087699 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:19:10.087652 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls podName:5a67e604-f9bc-46ad-8d26-c13a2465c56b nodeName:}" failed. No retries permitted until 2026-07-12 22:19:10.587631828 +0000 UTC m=+1599.285119386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls") pod "maas-api-e2e-ait-5249e6ec-866595574f-9296h" (UID: "5a67e604-f9bc-46ad-8d26-c13a2465c56b") : secret "maas-api-e2e-ait-5249e6ec-serving-cert" not found Jul 12 22:19:10.088236 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.088190 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.097298 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.097262 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qjjj\" (UniqueName: \"kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.593025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.592985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:10.593242 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:19:10.593120 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-serving-cert: secret "maas-api-e2e-ait-5249e6ec-serving-cert" not found Jul 12 22:19:10.593242 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:19:10.593186 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls podName:5a67e604-f9bc-46ad-8d26-c13a2465c56b nodeName:}" failed. No retries permitted until 2026-07-12 22:19:11.59317076 +0000 UTC m=+1600.290658319 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls") pod "maas-api-e2e-ait-5249e6ec-866595574f-9296h" (UID: "5a67e604-f9bc-46ad-8d26-c13a2465c56b") : secret "maas-api-e2e-ait-5249e6ec-serving-cert" not found Jul 12 22:19:10.946731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:10.946703 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:11.332139 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.332051 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:11.333151 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.333132 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:11.605183 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.605078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:11.607601 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.607564 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"maas-api-e2e-ait-5249e6ec-866595574f-9296h\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:11.753680 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.753634 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:11.904926 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.904901 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:19:11.907324 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:19:11.907294 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a67e604_f9bc_46ad_8d26_c13a2465c56b.slice/crio-714a4ab03a40c3361844090d662021ea3900dcbc4090f9e3a1c56f330601be0f WatchSource:0}: Error finding container 714a4ab03a40c3361844090d662021ea3900dcbc4090f9e3a1c56f330601be0f: Status 404 returned error can't find the container with id 714a4ab03a40c3361844090d662021ea3900dcbc4090f9e3a1c56f330601be0f Jul 12 22:19:11.908777 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:11.908755 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:19:12.338839 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:12.338742 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" event={"ID":"5a67e604-f9bc-46ad-8d26-c13a2465c56b","Type":"ContainerStarted","Data":"714a4ab03a40c3361844090d662021ea3900dcbc4090f9e3a1c56f330601be0f"} Jul 12 22:19:13.346017 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:13.345978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" event={"ID":"5a67e604-f9bc-46ad-8d26-c13a2465c56b","Type":"ContainerStarted","Data":"94b1eb8f8719a977bb731c3f34d0b5830c83f26996969796de34046523d6af06"} Jul 12 22:19:13.346484 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:13.346237 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:13.366687 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:13.366619 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" podStartSLOduration=3.778052196 podStartE2EDuration="4.366594861s" totalImagesPullingTime="588.542665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:19:13.365521992 +0000 UTC m=+1602.063009597" watchObservedRunningTime="2026-07-12 22:19:13.366594861 +0000 UTC m=+1602.064082444" Jul 12 22:19:19.356845 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:19.356812 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:19:26.334241 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.334175 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 12 22:19:26.334659 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.334281 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 12 22:19:26.427834 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.427797 2575 generic.go:356] "Generic (PLEG): container finished" podID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerID="52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180" exitCode=0 Jul 12 22:19:26.427976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.427871 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerDied","Data":"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180"} Jul 12 22:19:26.428383 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.428360 2575 scope.go:117] "RemoveContainer" containerID="52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180" Jul 12 22:19:26.429516 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.429483 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:26.429622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.429569 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:26.429686 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:26.429624 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:19:27.435481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:27.435442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerStarted","Data":"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6"} Jul 12 22:19:27.941403 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:27.941360 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:27.941655 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:27.941630 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 12 22:19:27.941741 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:27.941690 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 12 22:19:28.941644 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:28.941608 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 12 22:19:28.942045 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:28.941668 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 12 22:19:29.941141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:29.941104 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 12 22:19:29.941346 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:29.941158 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 12 22:19:30.941361 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:30.941324 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 12 22:19:30.941761 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:30.941379 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 12 22:19:31.945951 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:31.945919 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:31.946370 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:31.946220 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:19:31.947174 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:19:31.947156 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:20:15.474853 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.474816 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:20:15.475488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.475078 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" podUID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" containerName="maas-api" containerID="cri-o://94b1eb8f8719a977bb731c3f34d0b5830c83f26996969796de34046523d6af06" gracePeriod=30 Jul 12 22:20:15.694468 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.694427 2575 generic.go:356] "Generic (PLEG): container finished" podID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" containerID="94b1eb8f8719a977bb731c3f34d0b5830c83f26996969796de34046523d6af06" exitCode=0 Jul 12 22:20:15.694653 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.694541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" event={"ID":"5a67e604-f9bc-46ad-8d26-c13a2465c56b","Type":"ContainerDied","Data":"94b1eb8f8719a977bb731c3f34d0b5830c83f26996969796de34046523d6af06"} Jul 12 22:20:15.743303 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.743272 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:20:15.869073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.869034 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca\") pod \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " Jul 12 22:20:15.869301 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.869083 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") pod \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " Jul 12 22:20:15.869301 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.869235 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qjjj\" (UniqueName: \"kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj\") pod \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\" (UID: \"5a67e604-f9bc-46ad-8d26-c13a2465c56b\") " Jul 12 22:20:15.869454 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.869427 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a67e604-f9bc-46ad-8d26-c13a2465c56b" (UID: "5a67e604-f9bc-46ad-8d26-c13a2465c56b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:20:15.871466 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.871433 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a67e604-f9bc-46ad-8d26-c13a2465c56b" (UID: "5a67e604-f9bc-46ad-8d26-c13a2465c56b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:20:15.871466 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.871451 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj" (OuterVolumeSpecName: "kube-api-access-5qjjj") pod "5a67e604-f9bc-46ad-8d26-c13a2465c56b" (UID: "5a67e604-f9bc-46ad-8d26-c13a2465c56b"). InnerVolumeSpecName "kube-api-access-5qjjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:20:15.970448 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.970402 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qjjj\" (UniqueName: \"kubernetes.io/projected/5a67e604-f9bc-46ad-8d26-c13a2465c56b-kube-api-access-5qjjj\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:15.970448 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.970442 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a67e604-f9bc-46ad-8d26-c13a2465c56b-openshift-service-ca\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:15.970448 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:15.970453 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a67e604-f9bc-46ad-8d26-c13a2465c56b-maas-api-tls\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:16.702652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:16.702614 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" event={"ID":"5a67e604-f9bc-46ad-8d26-c13a2465c56b","Type":"ContainerDied","Data":"714a4ab03a40c3361844090d662021ea3900dcbc4090f9e3a1c56f330601be0f"} Jul 12 22:20:16.702652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:16.702651 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h" Jul 12 22:20:16.703147 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:16.702676 2575 scope.go:117] "RemoveContainer" containerID="94b1eb8f8719a977bb731c3f34d0b5830c83f26996969796de34046523d6af06" Jul 12 22:20:16.733979 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:16.733949 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:20:16.735918 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:16.735895 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5249e6ec-866595574f-9296h"] Jul 12 22:20:17.789295 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:17.789265 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" path="/var/lib/kubelet/pods/5a67e604-f9bc-46ad-8d26-c13a2465c56b/volumes" Jul 12 22:20:30.064174 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.064128 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:20:30.064755 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.064449 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" containerID="cri-o://8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6" gracePeriod=30 Jul 12 22:20:30.736387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.736324 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:20:30.736920 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.736895 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" containerName="maas-api" Jul 12 22:20:30.736920 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.736917 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" containerName="maas-api" Jul 12 22:20:30.737094 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.737065 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a67e604-f9bc-46ad-8d26-c13a2465c56b" containerName="maas-api" Jul 12 22:20:30.741602 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.741574 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.745042 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.745021 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-dccf31a6-gw-openshift-default-dockercfg-bptxx\"" Jul 12 22:20:30.768013 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.767978 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:20:30.811931 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.811880 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4sg6\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.811954 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.811984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812143 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812476 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812166 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.812476 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.812336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913401 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913599 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913599 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913599 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913599 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4sg6\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913640 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.913822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913719 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.914030 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.913880 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.914086 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.914060 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.914157 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.914097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.914484 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.914463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.915869 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.915849 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.915970 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.915948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.922107 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.922072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:30.922277 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:30.922251 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4sg6\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6\") pod \"e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:31.052699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.052601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:31.213976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.213948 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:20:31.215906 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:20:31.215877 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5072765_cb31_41ee_b513_63e059eee279.slice/crio-81ab3a6283db28b082bd40c2f782a37d0152765eac53de787b45cccae510282d WatchSource:0}: Error finding container 81ab3a6283db28b082bd40c2f782a37d0152765eac53de787b45cccae510282d: Status 404 returned error can't find the container with id 81ab3a6283db28b082bd40c2f782a37d0152765eac53de787b45cccae510282d Jul 12 22:20:31.218092 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.218059 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:31.218219 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.218125 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:31.218219 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.218168 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:31.790274 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.790236 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerStarted","Data":"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8"} Jul 12 22:20:31.790478 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.790288 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerStarted","Data":"81ab3a6283db28b082bd40c2f782a37d0152765eac53de787b45cccae510282d"} Jul 12 22:20:31.814189 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.814122 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podStartSLOduration=1.814106258 podStartE2EDuration="1.814106258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:20:31.812155803 +0000 UTC m=+1680.509643375" watchObservedRunningTime="2026-07-12 22:20:31.814106258 +0000 UTC m=+1680.511593836" Jul 12 22:20:31.949634 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.949595 2575 patch_prober.go:28] interesting pod/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 22:20:31.949832 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:31.949674 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:20:32.052965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:32.052865 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:32.059180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:32.059151 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:32.797524 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:32.797494 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:32.798523 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:32.798504 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:35.340256 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.340160 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:20:35.460985 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.460939 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lm75\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.460985 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.460993 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461013 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461037 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461081 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461107 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461142 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461176 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461223 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket\") pod \"c0756510-b55c-41e5-b2ab-fbe1de9070de\" (UID: \"c0756510-b55c-41e5-b2ab-fbe1de9070de\") " Jul 12 22:20:35.461605 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461440 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:20:35.461605 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461506 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:20:35.461605 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461541 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.461761 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461644 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:20:35.461761 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.461690 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:20:35.462059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.462031 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:20:35.463400 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.463369 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:20:35.463400 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.463379 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:20:35.463575 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.463398 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75" (OuterVolumeSpecName: "kube-api-access-4lm75") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "kube-api-access-4lm75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:20:35.463575 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.463439 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0756510-b55c-41e5-b2ab-fbe1de9070de" (UID: "c0756510-b55c-41e5-b2ab-fbe1de9070de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:20:35.563125 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563090 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0756510-b55c-41e5-b2ab-fbe1de9070de-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563125 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563123 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563125 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563131 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563139 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563148 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lm75\" (UniqueName: \"kubernetes.io/projected/c0756510-b55c-41e5-b2ab-fbe1de9070de-kube-api-access-4lm75\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563158 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563166 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.563387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.563176 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0756510-b55c-41e5-b2ab-fbe1de9070de-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:20:35.815792 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.815759 2575 generic.go:356] "Generic (PLEG): container finished" podID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerID="8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6" exitCode=0 Jul 12 22:20:35.815976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.815817 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" Jul 12 22:20:35.815976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.815849 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerDied","Data":"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6"} Jul 12 22:20:35.815976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.815906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r" event={"ID":"c0756510-b55c-41e5-b2ab-fbe1de9070de","Type":"ContainerDied","Data":"aca87d03db8ffd98728fa1312e5c3f6a8e64287f6b670856b9c5c95a315c3157"} Jul 12 22:20:35.815976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.815928 2575 scope.go:117] "RemoveContainer" containerID="8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6" Jul 12 22:20:35.827437 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.827415 2575 scope.go:117] "RemoveContainer" containerID="52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180" Jul 12 22:20:35.836068 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.836033 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:20:35.837048 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.837017 2575 scope.go:117] "RemoveContainer" containerID="8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6" Jul 12 22:20:35.837505 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:20:35.837480 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6\": container with ID starting with 8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6 not found: ID does not exist" containerID="8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6" Jul 12 22:20:35.837630 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.837521 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6"} err="failed to get container status \"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6\": rpc error: code = NotFound desc = could not find container \"8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6\": container with ID starting with 8f856bef62251330d0aa6efbd2971e9d7d966939dbde37fa5e673ddfb3c902c6 not found: ID does not exist" Jul 12 22:20:35.837630 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.837547 2575 scope.go:117] "RemoveContainer" containerID="52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180" Jul 12 22:20:35.837886 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:20:35.837864 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180\": container with ID starting with 52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180 not found: ID does not exist" containerID="52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180" Jul 12 22:20:35.837932 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.837894 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180"} err="failed to get container status \"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180\": rpc error: code = NotFound desc = could not find container \"52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180\": container with ID starting with 52a543881c57c480853a4282a706c6b58788a1691ff9cd0401197ad46b95a180 not found: ID does not exist" Jul 12 22:20:35.838830 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:35.838802 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5249e6ec-openshift-default-6c55776df7-7b42r"] Jul 12 22:20:37.789202 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:37.789167 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" path="/var/lib/kubelet/pods/c0756510-b55c-41e5-b2ab-fbe1de9070de/volumes" Jul 12 22:20:47.798954 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.798920 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 12 22:20:47.799374 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.798987 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 12 22:20:47.882616 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.882585 2575 generic.go:356] "Generic (PLEG): container finished" podID="d5072765-cb31-41ee-b513-63e059eee279" containerID="00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8" exitCode=0 Jul 12 22:20:47.882786 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.882644 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerDied","Data":"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8"} Jul 12 22:20:47.883025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.883012 2575 scope.go:117] "RemoveContainer" containerID="00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8" Jul 12 22:20:47.884090 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.884052 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:47.884238 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.884122 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:47.884238 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:47.884153 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:20:48.893431 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:48.893384 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerStarted","Data":"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f"} Jul 12 22:20:49.052900 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:49.052867 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:49.053121 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:49.053099 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 12 22:20:49.053182 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:49.053152 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 12 22:20:50.053195 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:50.053152 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 12 22:20:50.053584 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:50.053201 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 12 22:20:51.053978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:51.053939 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 12 22:20:51.054444 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:51.053996 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 12 22:20:52.053689 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:52.053645 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 12 22:20:52.053877 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:52.053711 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 12 22:20:53.057097 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:53.057061 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:53.057510 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:53.057440 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:20:53.058081 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:20:53.058060 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:21:36.537474 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:36.537438 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:21:36.537972 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:36.537651 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" containerID="cri-o://10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f" gracePeriod=30 Jul 12 22:21:38.059454 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:38.059405 2575 patch_prober.go:28] interesting pod/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 12 22:21:38.059854 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:38.059488 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:21:41.816473 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.816405 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:21:41.913789 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913757 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4sg6\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.913978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913802 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.913978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913823 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.913978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913887 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.913978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913904 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.913978 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.913933 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.914264 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914010 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.914264 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914062 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.914264 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914083 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket\") pod \"d5072765-cb31-41ee-b513-63e059eee279\" (UID: \"d5072765-cb31-41ee-b513-63e059eee279\") " Jul 12 22:21:41.914264 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914155 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:21:41.914544 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914521 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:41.914631 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914553 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:21:41.914631 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914573 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:21:41.914953 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.914923 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:21:41.915188 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.915076 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:21:41.916177 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.916152 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:21:41.916486 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.916453 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6" (OuterVolumeSpecName: "kube-api-access-v4sg6") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "kube-api-access-v4sg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:21:41.916486 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.916475 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:21:41.916905 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:41.916882 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5072765-cb31-41ee-b513-63e059eee279" (UID: "d5072765-cb31-41ee-b513-63e059eee279"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:21:42.015198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015167 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015194 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015219 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5072765-cb31-41ee-b513-63e059eee279-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015227 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5072765-cb31-41ee-b513-63e059eee279-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015235 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015528 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015242 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015528 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015250 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4sg6\" (UniqueName: \"kubernetes.io/projected/d5072765-cb31-41ee-b513-63e059eee279-kube-api-access-v4sg6\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.015528 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.015259 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5072765-cb31-41ee-b513-63e059eee279-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:21:42.178600 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.178559 2575 generic.go:356] "Generic (PLEG): container finished" podID="d5072765-cb31-41ee-b513-63e059eee279" containerID="10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f" exitCode=0 Jul 12 22:21:42.178769 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.178607 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" Jul 12 22:21:42.178769 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.178653 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerDied","Data":"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f"} Jul 12 22:21:42.178769 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.178718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2" event={"ID":"d5072765-cb31-41ee-b513-63e059eee279","Type":"ContainerDied","Data":"81ab3a6283db28b082bd40c2f782a37d0152765eac53de787b45cccae510282d"} Jul 12 22:21:42.178769 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.178748 2575 scope.go:117] "RemoveContainer" containerID="10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f" Jul 12 22:21:42.188374 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.188359 2575 scope.go:117] "RemoveContainer" containerID="00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8" Jul 12 22:21:42.196804 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.196787 2575 scope.go:117] "RemoveContainer" containerID="10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f" Jul 12 22:21:42.197036 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:21:42.197021 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f\": container with ID starting with 10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f not found: ID does not exist" containerID="10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f" Jul 12 22:21:42.197079 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.197045 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f"} err="failed to get container status \"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f\": rpc error: code = NotFound desc = could not find container \"10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f\": container with ID starting with 10fb9c25992340f63976e55598845b7f2e8b4a6a89d48f2cc5c9345e5f5eef8f not found: ID does not exist" Jul 12 22:21:42.197079 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.197063 2575 scope.go:117] "RemoveContainer" containerID="00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8" Jul 12 22:21:42.197310 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:21:42.197292 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8\": container with ID starting with 00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8 not found: ID does not exist" containerID="00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8" Jul 12 22:21:42.197366 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.197314 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8"} err="failed to get container status \"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8\": rpc error: code = NotFound desc = could not find container \"00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8\": container with ID starting with 00d147eff2570c2b741534997a71da860c972346f24122a1c956957d0efc08e8 not found: ID does not exist" Jul 12 22:21:42.201671 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.201640 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:21:42.205706 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.205685 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-dccf31a6-gw-openshift-default-7c744fc799-wxvn2"] Jul 12 22:21:42.933446 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.933414 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:21:42.933969 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.933947 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.934092 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934068 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.934169 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934105 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934169 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934114 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934169 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934127 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934169 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934136 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934173 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934181 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934334 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934349 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0756510-b55c-41e5-b2ab-fbe1de9070de" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934364 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.934408 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.934372 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5072765-cb31-41ee-b513-63e059eee279" containerName="istio-proxy" Jul 12 22:21:42.939351 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.939330 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:42.941879 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.941861 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-92b395e7-openshift-default-dockercfg-rh7mv\"" Jul 12 22:21:42.949382 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:42.949357 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:21:43.025876 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.025839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026101 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.025899 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026101 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.025917 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026101 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.025932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026101 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.026028 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026101 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.026067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.026124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.026187 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phsdv\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.026397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.026248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127194 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127279 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127301 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127351 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127404 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phsdv\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127583 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127679 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127946 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127927 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.127993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.127967 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.128168 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.128151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.129626 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.129601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.129825 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.129807 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.137622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.137594 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.137732 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.137685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phsdv\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv\") pod \"e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.252340 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.252242 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:43.394629 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.394605 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:21:43.397014 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:21:43.396983 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73aa7d56_6d4a_4c22_83a1_991cbbbb63ba.slice/crio-ed4f6fb2467d8fac3b00f92a8984a1323060bf251714a719629a380060431f78 WatchSource:0}: Error finding container ed4f6fb2467d8fac3b00f92a8984a1323060bf251714a719629a380060431f78: Status 404 returned error can't find the container with id ed4f6fb2467d8fac3b00f92a8984a1323060bf251714a719629a380060431f78 Jul 12 22:21:43.399424 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.399382 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:43.399511 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.399471 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:43.399591 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.399518 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:43.789304 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:43.789271 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5072765-cb31-41ee-b513-63e059eee279" path="/var/lib/kubelet/pods/d5072765-cb31-41ee-b513-63e059eee279/volumes" Jul 12 22:21:44.196295 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:44.196256 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerStarted","Data":"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b"} Jul 12 22:21:44.196780 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:44.196329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerStarted","Data":"ed4f6fb2467d8fac3b00f92a8984a1323060bf251714a719629a380060431f78"} Jul 12 22:21:44.219266 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:44.219186 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podStartSLOduration=2.219172301 podStartE2EDuration="2.219172301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:21:44.218373561 +0000 UTC m=+1752.915861142" watchObservedRunningTime="2026-07-12 22:21:44.219172301 +0000 UTC m=+1752.916659880" Jul 12 22:21:44.253109 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:44.253072 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:44.257813 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:44.257785 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:45.203661 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:45.203627 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:45.204507 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:45.204490 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:56.259201 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.259166 2575 generic.go:356] "Generic (PLEG): container finished" podID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerID="8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b" exitCode=0 Jul 12 22:21:56.259622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.259247 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerDied","Data":"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b"} Jul 12 22:21:56.259733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.259716 2575 scope.go:117] "RemoveContainer" containerID="8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b" Jul 12 22:21:56.260744 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.260717 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:56.260790 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.260782 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:56.260828 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:56.260812 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:21:57.266992 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:57.266959 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerStarted","Data":"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7"} Jul 12 22:21:58.253461 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:58.253416 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:21:58.254504 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:58.254479 2575 patch_prober.go:28] interesting pod/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 12 22:21:58.254611 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:58.254539 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 12 22:21:59.253192 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:59.253151 2575 patch_prober.go:28] interesting pod/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 12 22:21:59.253700 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:21:59.253221 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 12 22:22:00.204385 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:22:00.204330 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:22:00.253073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:22:00.253035 2575 patch_prober.go:28] interesting pod/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 12 22:22:00.253283 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:22:00.253106 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 12 22:22:01.257486 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:22:01.257452 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:22:01.258649 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:22:01.258629 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:23:15.262812 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.262772 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:23:15.263437 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.263058 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" containerID="cri-o://7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7" gracePeriod=30 Jul 12 22:23:15.575152 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.575065 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:23:15.578980 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.578937 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.582309 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.582276 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6cea543a-openshift-default-dockercfg-wt4dz\"" Jul 12 22:23:15.592039 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.592001 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:23:15.741794 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741749 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.741794 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741799 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741827 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741909 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7lqh\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.741935 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742288 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.742048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742288 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.742089 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.742288 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.742129 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843434 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843335 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843434 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7lqh\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843479 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843588 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843856 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843670 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843856 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843705 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.843856 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843735 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.844019 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843959 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.844019 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.843994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.844130 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.844093 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.844226 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.844187 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.844434 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.844416 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.845733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.845712 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.846023 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.846003 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.853363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.853333 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.853515 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.853498 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7lqh\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh\") pod \"e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:15.893265 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:15.893227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:16.050707 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.050679 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:23:16.053427 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:23:16.053396 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd333b0be_d814_4a28_afe4_874fd9bbb8fe.slice/crio-d0f1a432693abbaeb04701788fd7a46f5e515d27ee3a9999f3bba78cd54b2a01 WatchSource:0}: Error finding container d0f1a432693abbaeb04701788fd7a46f5e515d27ee3a9999f3bba78cd54b2a01: Status 404 returned error can't find the container with id d0f1a432693abbaeb04701788fd7a46f5e515d27ee3a9999f3bba78cd54b2a01 Jul 12 22:23:16.055686 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.055652 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:16.055783 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.055725 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:16.055783 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.055755 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:16.260699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.260657 2575 patch_prober.go:28] interesting pod/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 22:23:16.260862 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.260728 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:23:16.690063 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.690027 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerStarted","Data":"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5"} Jul 12 22:23:16.690534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.690125 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerStarted","Data":"d0f1a432693abbaeb04701788fd7a46f5e515d27ee3a9999f3bba78cd54b2a01"} Jul 12 22:23:16.730221 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.730124 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podStartSLOduration=1.730100019 podStartE2EDuration="1.730100019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:23:16.727440091 +0000 UTC m=+1845.424927671" watchObservedRunningTime="2026-07-12 22:23:16.730100019 +0000 UTC m=+1845.427587600" Jul 12 22:23:16.894160 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:16.894113 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:17.899375 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:17.899340 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:18.701069 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:18.701035 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:18.702066 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:18.702047 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:20.532710 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.532200 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604023 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604080 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phsdv\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604130 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604165 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604248 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604301 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604351 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604396 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.604502 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert\") pod \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\" (UID: \"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba\") " Jul 12 22:23:20.605994 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.605964 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:23:20.607027 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.606170 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:23:20.607027 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.606346 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:23:20.607027 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.606874 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:23:20.607287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.607065 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:23:20.615114 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.615064 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:23:20.616673 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.616633 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:23:20.616876 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.616740 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:23:20.617146 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.617121 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv" (OuterVolumeSpecName: "kube-api-access-phsdv") pod "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" (UID: "73aa7d56-6d4a-4c22-83a1-991cbbbb63ba"). InnerVolumeSpecName "kube-api-access-phsdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707020 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707072 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707086 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phsdv\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-kube-api-access-phsdv\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707099 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707112 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707125 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707140 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707153 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.707248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.707165 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:23:20.714198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.714163 2575 generic.go:356] "Generic (PLEG): container finished" podID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerID="7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7" exitCode=0 Jul 12 22:23:20.714388 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.714234 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" Jul 12 22:23:20.714388 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.714237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerDied","Data":"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7"} Jul 12 22:23:20.714388 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.714291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc" event={"ID":"73aa7d56-6d4a-4c22-83a1-991cbbbb63ba","Type":"ContainerDied","Data":"ed4f6fb2467d8fac3b00f92a8984a1323060bf251714a719629a380060431f78"} Jul 12 22:23:20.714388 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.714316 2575 scope.go:117] "RemoveContainer" containerID="7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7" Jul 12 22:23:20.724975 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.724953 2575 scope.go:117] "RemoveContainer" containerID="8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b" Jul 12 22:23:20.734052 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.734033 2575 scope.go:117] "RemoveContainer" containerID="7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7" Jul 12 22:23:20.734546 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:23:20.734522 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7\": container with ID starting with 7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7 not found: ID does not exist" containerID="7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7" Jul 12 22:23:20.734660 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.734554 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7"} err="failed to get container status \"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7\": rpc error: code = NotFound desc = could not find container \"7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7\": container with ID starting with 7448a1b27f6fa97a6cc7b62bb483e3ec8efe35002ccd179a1f253c0768d895b7 not found: ID does not exist" Jul 12 22:23:20.734660 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.734575 2575 scope.go:117] "RemoveContainer" containerID="8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b" Jul 12 22:23:20.734933 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:23:20.734887 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b\": container with ID starting with 8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b not found: ID does not exist" containerID="8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b" Jul 12 22:23:20.735031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.734925 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b"} err="failed to get container status \"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b\": rpc error: code = NotFound desc = could not find container \"8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b\": container with ID starting with 8182af013a425c5759cab1ca3f10dd74e66d5f7746c60b8979bbd721adf8597b not found: ID does not exist" Jul 12 22:23:20.737976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.737944 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:23:20.742006 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:20.741977 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-92b395e7-openshift-default-5d7fdbfdcf-287lc"] Jul 12 22:23:21.803617 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:21.803578 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" path="/var/lib/kubelet/pods/73aa7d56-6d4a-4c22-83a1-991cbbbb63ba/volumes" Jul 12 22:23:27.754539 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.754506 2575 generic.go:356] "Generic (PLEG): container finished" podID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerID="61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5" exitCode=0 Jul 12 22:23:27.754955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.754575 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerDied","Data":"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5"} Jul 12 22:23:27.755086 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.755069 2575 scope.go:117] "RemoveContainer" containerID="61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5" Jul 12 22:23:27.756109 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.756081 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:27.756189 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.756174 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:27.756250 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:27.756230 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:23:28.763487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:28.763451 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerStarted","Data":"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d"} Jul 12 22:23:28.894733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:28.894698 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:28.895365 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:28.895331 2575 patch_prober.go:28] interesting pod/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 12 22:23:28.895510 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:28.895396 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 12 22:23:29.894799 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:29.894746 2575 patch_prober.go:28] interesting pod/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 12 22:23:29.895348 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:29.894839 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 12 22:23:30.893827 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:30.893776 2575 patch_prober.go:28] interesting pod/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 12 22:23:30.894058 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:30.893853 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 12 22:23:31.894859 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:31.894344 2575 patch_prober.go:28] interesting pod/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 12 22:23:31.894859 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:31.894439 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 12 22:23:32.894008 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:32.893962 2575 patch_prober.go:28] interesting pod/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 12 22:23:32.894266 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:32.894036 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 12 22:23:33.701344 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:33.701307 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:33.897812 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:33.897778 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:23:33.898703 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:23:33.898681 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:24:36.400446 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:36.400405 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:24:36.400900 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:36.400630 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" containerID="cri-o://6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d" gracePeriod=30 Jul 12 22:24:41.661354 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.661327 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:24:41.696916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696788 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.696916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696833 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7lqh\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.696916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696873 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.696916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696905 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.697287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696927 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.697287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.696980 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.697287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.697006 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.697287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.697040 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.697287 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.697067 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs\") pod \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\" (UID: \"d333b0be-d814-4a28-afe4-874fd9bbb8fe\") " Jul 12 22:24:41.699201 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.698000 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:24:41.699201 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.698999 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:24:41.699946 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.699709 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:24:41.699946 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.699756 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:24:41.700114 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.700003 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:24:41.701532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.700625 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:24:41.701532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.700626 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:24:41.701679 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.701610 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh" (OuterVolumeSpecName: "kube-api-access-c7lqh") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "kube-api-access-c7lqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:24:41.709199 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.709170 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d333b0be-d814-4a28-afe4-874fd9bbb8fe" (UID: "d333b0be-d814-4a28-afe4-874fd9bbb8fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:24:41.799198 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799169 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799255 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799271 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7lqh\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-kube-api-access-c7lqh\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799287 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799302 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799314 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799333 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799326 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d333b0be-d814-4a28-afe4-874fd9bbb8fe-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799518 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799339 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:41.799518 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:41.799351 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d333b0be-d814-4a28-afe4-874fd9bbb8fe-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:24:42.160682 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.160645 2575 generic.go:356] "Generic (PLEG): container finished" podID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerID="6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d" exitCode=0 Jul 12 22:24:42.160862 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.160721 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" Jul 12 22:24:42.160862 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.160719 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerDied","Data":"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d"} Jul 12 22:24:42.160862 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.160784 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn" event={"ID":"d333b0be-d814-4a28-afe4-874fd9bbb8fe","Type":"ContainerDied","Data":"d0f1a432693abbaeb04701788fd7a46f5e515d27ee3a9999f3bba78cd54b2a01"} Jul 12 22:24:42.160862 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.160808 2575 scope.go:117] "RemoveContainer" containerID="6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d" Jul 12 22:24:42.170015 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.169993 2575 scope.go:117] "RemoveContainer" containerID="61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5" Jul 12 22:24:42.179487 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.179466 2575 scope.go:117] "RemoveContainer" containerID="6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d" Jul 12 22:24:42.179817 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:24:42.179796 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d\": container with ID starting with 6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d not found: ID does not exist" containerID="6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d" Jul 12 22:24:42.179892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.179834 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d"} err="failed to get container status \"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d\": rpc error: code = NotFound desc = could not find container \"6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d\": container with ID starting with 6c271bb3d2eb51e11f2d80107c8d141a4dfda0fd44aa089bdd988bf7d6db374d not found: ID does not exist" Jul 12 22:24:42.179892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.179853 2575 scope.go:117] "RemoveContainer" containerID="61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5" Jul 12 22:24:42.180163 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:24:42.180140 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5\": container with ID starting with 61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5 not found: ID does not exist" containerID="61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5" Jul 12 22:24:42.180230 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.180174 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5"} err="failed to get container status \"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5\": rpc error: code = NotFound desc = could not find container \"61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5\": container with ID starting with 61334e84c62e4b140d4979b1896f7b916c91979453617a67aab6eb9f1589cca5 not found: ID does not exist" Jul 12 22:24:42.182523 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.182411 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:24:42.186092 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:42.186062 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6cea543a-openshift-default-75bf746f6c-qjqmn"] Jul 12 22:24:43.789107 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:24:43.789069 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" path="/var/lib/kubelet/pods/d333b0be-d814-4a28-afe4-874fd9bbb8fe/volumes" Jul 12 22:26:12.700385 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.700338 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:12.701019 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701001 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:26:12.701019 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701020 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:26:12.701137 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701037 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:26:12.701137 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701043 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:26:12.701137 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701130 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:26:12.701272 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701139 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:26:12.701272 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.701147 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:26:12.704158 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.704141 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.707009 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.706986 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3283879f-openshift-default-dockercfg-qr889\"" Jul 12 22:26:12.726324 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.726277 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:12.772104 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772298 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772298 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772298 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8kzv\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772411 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772411 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772322 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772411 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772411 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772405 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.772536 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.772424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.873892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.873797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.873892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.873848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8kzv\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.873892 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.873874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874117 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874180 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874148 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874588 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874559 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874612 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874729 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874673 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.874868 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.875005 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.874982 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.876650 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.876624 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.876756 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.876618 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.889993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.889960 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:12.890687 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:12.890665 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8kzv\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv\") pod \"e2e-mt-3283879f-openshift-default-6d76666599-69frp\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:13.016306 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.016174 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:13.176753 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.176582 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:13.179321 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:26:13.179290 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb25726_e353_45ff_8393_f280bb90c4e2.slice/crio-1c4b213ac0001474467b88ba85c0fc5f423b4b13f1e05a91ee6e47b9522b5799 WatchSource:0}: Error finding container 1c4b213ac0001474467b88ba85c0fc5f423b4b13f1e05a91ee6e47b9522b5799: Status 404 returned error can't find the container with id 1c4b213ac0001474467b88ba85c0fc5f423b4b13f1e05a91ee6e47b9522b5799 Jul 12 22:26:13.181931 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.181895 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:13.182010 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.181977 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:13.182048 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.182011 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:13.667190 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.667154 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerStarted","Data":"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525"} Jul 12 22:26:13.667434 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.667230 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerStarted","Data":"1c4b213ac0001474467b88ba85c0fc5f423b4b13f1e05a91ee6e47b9522b5799"} Jul 12 22:26:13.690911 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:13.690849 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podStartSLOduration=1.6908169229999999 podStartE2EDuration="1.690816923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:26:13.689919889 +0000 UTC m=+2022.387407472" watchObservedRunningTime="2026-07-12 22:26:13.690816923 +0000 UTC m=+2022.388304510" Jul 12 22:26:14.017412 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:14.017320 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:14.022366 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:14.022337 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:14.673080 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:14.673041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:14.674155 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:14.674128 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:30.716382 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:30.716336 2575 patch_prober.go:28] interesting pod/e2e-mt-3283879f-openshift-default-6d76666599-69frp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 22:26:30.716793 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:30.716443 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 22:26:31.770220 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.770182 2575 generic.go:356] "Generic (PLEG): container finished" podID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerID="28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525" exitCode=0 Jul 12 22:26:31.770645 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.770254 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerDied","Data":"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525"} Jul 12 22:26:31.770715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.770701 2575 scope.go:117] "RemoveContainer" containerID="28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525" Jul 12 22:26:31.771786 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.771758 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:31.771861 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.771845 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:31.771905 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:31.771889 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:26:32.777913 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:32.777879 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerStarted","Data":"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72"} Jul 12 22:26:33.017118 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:33.017074 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:33.017360 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:33.017336 2575 patch_prober.go:28] interesting pod/e2e-mt-3283879f-openshift-default-6d76666599-69frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 12 22:26:33.017410 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:33.017389 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 12 22:26:34.016718 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:34.016677 2575 patch_prober.go:28] interesting pod/e2e-mt-3283879f-openshift-default-6d76666599-69frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 12 22:26:34.017117 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:34.016733 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 12 22:26:35.017625 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:35.017582 2575 patch_prober.go:28] interesting pod/e2e-mt-3283879f-openshift-default-6d76666599-69frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 12 22:26:35.018035 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:35.017652 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 12 22:26:36.017512 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:36.017472 2575 patch_prober.go:28] interesting pod/e2e-mt-3283879f-openshift-default-6d76666599-69frp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 12 22:26:36.017695 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:36.017526 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 12 22:26:37.020762 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:37.020731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:37.021157 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:37.021090 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:37.021922 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:37.021903 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:53.070756 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:53.070663 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:53.071380 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:53.070914 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" containerID="cri-o://40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72" gracePeriod=30 Jul 12 22:26:58.339708 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.339630 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:58.441037 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.440994 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441037 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441043 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441079 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441100 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441118 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441163 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441231 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441262 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8kzv\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441593 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441338 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:26:58.441593 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441369 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs\") pod \"0fb25726-e353-45ff-8393-f280bb90c4e2\" (UID: \"0fb25726-e353-45ff-8393-f280bb90c4e2\") " Jul 12 22:26:58.441593 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441562 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:26:58.441725 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441603 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:26:58.441725 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441613 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:26:58.441833 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441767 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.441833 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441780 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.441833 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441789 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fb25726-e353-45ff-8393-f280bb90c4e2-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.441833 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.441796 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.442254 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.442222 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:26:58.443393 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.443366 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:26:58.443585 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.443565 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:26:58.443650 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.443594 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv" (OuterVolumeSpecName: "kube-api-access-m8kzv") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "kube-api-access-m8kzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:26:58.443650 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.443624 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0fb25726-e353-45ff-8393-f280bb90c4e2" (UID: "0fb25726-e353-45ff-8393-f280bb90c4e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:26:58.542500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.542458 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.542500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.542490 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.542500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.542499 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.542500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.542506 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.542766 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.542515 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8kzv\" (UniqueName: \"kubernetes.io/projected/0fb25726-e353-45ff-8393-f280bb90c4e2-kube-api-access-m8kzv\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:26:58.930454 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.930414 2575 generic.go:356] "Generic (PLEG): container finished" podID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerID="40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72" exitCode=0 Jul 12 22:26:58.930635 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.930492 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" Jul 12 22:26:58.930635 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.930503 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerDied","Data":"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72"} Jul 12 22:26:58.930635 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.930571 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp" event={"ID":"0fb25726-e353-45ff-8393-f280bb90c4e2","Type":"ContainerDied","Data":"1c4b213ac0001474467b88ba85c0fc5f423b4b13f1e05a91ee6e47b9522b5799"} Jul 12 22:26:58.930635 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.930596 2575 scope.go:117] "RemoveContainer" containerID="40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72" Jul 12 22:26:58.941022 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.941000 2575 scope.go:117] "RemoveContainer" containerID="28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525" Jul 12 22:26:58.950373 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.950350 2575 scope.go:117] "RemoveContainer" containerID="40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72" Jul 12 22:26:58.950820 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:26:58.950787 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72\": container with ID starting with 40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72 not found: ID does not exist" containerID="40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72" Jul 12 22:26:58.951152 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.950832 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72"} err="failed to get container status \"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72\": rpc error: code = NotFound desc = could not find container \"40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72\": container with ID starting with 40a6c0b966e78e39642192f3f78cd87dc472b1518220f147367e0478eb306c72 not found: ID does not exist" Jul 12 22:26:58.951152 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.950860 2575 scope.go:117] "RemoveContainer" containerID="28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525" Jul 12 22:26:58.951408 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:26:58.951381 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525\": container with ID starting with 28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525 not found: ID does not exist" containerID="28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525" Jul 12 22:26:58.951453 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.951417 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525"} err="failed to get container status \"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525\": rpc error: code = NotFound desc = could not find container \"28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525\": container with ID starting with 28b4ec1563348f6a34d038ee77fe3dc5443d8932687926887f82c544392be525 not found: ID does not exist" Jul 12 22:26:58.953623 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.953598 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:58.957968 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:58.957938 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3283879f-openshift-default-6d76666599-69frp"] Jul 12 22:26:59.788896 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:26:59.788856 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" path="/var/lib/kubelet/pods/0fb25726-e353-45ff-8393-f280bb90c4e2/volumes" Jul 12 22:27:12.971904 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.971866 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:27:12.972535 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972513 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972536 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972554 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972562 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972582 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972592 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972622 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:27:12.972640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972630 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aa7d56-6d4a-4c22-83a1-991cbbbb63ba" containerName="istio-proxy" Jul 12 22:27:12.972961 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972741 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:27:12.972961 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.972756 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d333b0be-d814-4a28-afe4-874fd9bbb8fe" containerName="istio-proxy" Jul 12 22:27:12.975181 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:12.975157 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.013002 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.012964 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d3e48a9a-openshift-default-dockercfg-6k46z\"" Jul 12 22:27:13.039969 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.039935 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:27:13.088642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088652 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088728 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088823 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjw92\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088913 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.088966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.088956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.189681 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.189651 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjw92\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.189894 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.189688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.189966 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.189946 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.189998 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190074 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190052 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190131 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190119 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190182 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190353 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190353 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190318 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190461 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190377 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190582 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190562 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190640 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.190749 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.190723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.192570 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.192542 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.192694 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.192676 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.204445 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.204410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.212547 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.212513 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjw92\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92\") pod \"e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.287420 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.287323 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:13.455697 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.455671 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:27:13.458458 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:27:13.458426 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc92a97_0d88_4a83_8e19_a95b7d74759e.slice/crio-2763cd957ec2e5c6977ba0d3ed24ee0974c60ecb58e98e359f8b63e054d710f7 WatchSource:0}: Error finding container 2763cd957ec2e5c6977ba0d3ed24ee0974c60ecb58e98e359f8b63e054d710f7: Status 404 returned error can't find the container with id 2763cd957ec2e5c6977ba0d3ed24ee0974c60ecb58e98e359f8b63e054d710f7 Jul 12 22:27:13.460623 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.460590 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:13.460711 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.460662 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:13.460711 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:13.460697 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:14.014271 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:14.014236 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerStarted","Data":"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4"} Jul 12 22:27:14.014667 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:14.014288 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerStarted","Data":"2763cd957ec2e5c6977ba0d3ed24ee0974c60ecb58e98e359f8b63e054d710f7"} Jul 12 22:27:14.047771 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:14.047719 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podStartSLOduration=2.047702715 podStartE2EDuration="2.047702715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:27:14.047461197 +0000 UTC m=+2082.744948778" watchObservedRunningTime="2026-07-12 22:27:14.047702715 +0000 UTC m=+2082.745190294" Jul 12 22:27:14.288601 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:14.288511 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:14.293701 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:14.293671 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:15.018936 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:15.018902 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:15.019849 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:15.019830 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:29.098197 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.098089 2575 generic.go:356] "Generic (PLEG): container finished" podID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerID="2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4" exitCode=0 Jul 12 22:27:29.098197 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.098137 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerDied","Data":"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4"} Jul 12 22:27:29.098652 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.098569 2575 scope.go:117] "RemoveContainer" containerID="2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4" Jul 12 22:27:29.099591 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.099563 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:29.099694 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.099629 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:29.099694 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:29.099663 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:27:30.019397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:30.019346 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:30.109950 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:30.109912 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerStarted","Data":"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1"} Jul 12 22:27:30.287670 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:30.287569 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:30.288958 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:30.288927 2575 patch_prober.go:28] interesting pod/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 12 22:27:30.289097 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:30.288993 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 12 22:27:31.288769 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:31.288727 2575 patch_prober.go:28] interesting pod/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 12 22:27:31.289936 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:31.289522 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 12 22:27:32.288840 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:32.288795 2575 patch_prober.go:28] interesting pod/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 12 22:27:32.289481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:32.289445 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 12 22:27:33.288145 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:33.288098 2575 patch_prober.go:28] interesting pod/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 12 22:27:33.288386 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:33.288195 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 12 22:27:34.292385 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:34.292353 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:34.292782 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:34.292642 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:27:34.293486 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:27:34.293467 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:28:55.899552 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:28:55.899511 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:28:56.586614 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:28:56.586572 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" containerID="cri-o://3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1" gracePeriod=30 Jul 12 22:29:01.863177 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.863093 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:29:01.993740 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993695 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.993955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993764 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.993955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993793 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.993955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993840 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.993955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993884 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.993955 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993924 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.994276 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.993986 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjw92\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.994276 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994029 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.994276 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994069 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data\") pod \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\" (UID: \"4bc92a97-0d88-4a83-8e19-a95b7d74759e\") " Jul 12 22:29:01.994276 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994174 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:29:01.994481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994325 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:29:01.994632 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994597 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:29:01.994688 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994613 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:01.994731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994695 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:01.994731 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.994696 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:29:01.995233 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.995175 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data" (OuterVolumeSpecName: "istio-data") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:29:01.996328 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.996298 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:29:01.996532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.996510 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token" (OuterVolumeSpecName: "istio-token") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:29:01.996588 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.996538 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92" (OuterVolumeSpecName: "kube-api-access-bjw92") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "kube-api-access-bjw92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:29:01.996838 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:01.996822 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4bc92a97-0d88-4a83-8e19-a95b7d74759e" (UID: "4bc92a97-0d88-4a83-8e19-a95b7d74759e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:29:02.095532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095477 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095514 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095528 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095540 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjw92\" (UniqueName: \"kubernetes.io/projected/4bc92a97-0d88-4a83-8e19-a95b7d74759e-kube-api-access-bjw92\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095553 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095564 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.095810 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.095575 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bc92a97-0d88-4a83-8e19-a95b7d74759e-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:29:02.623303 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.623270 2575 generic.go:356] "Generic (PLEG): container finished" podID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerID="3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1" exitCode=0 Jul 12 22:29:02.623488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.623329 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" Jul 12 22:29:02.623488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.623350 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerDied","Data":"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1"} Jul 12 22:29:02.623488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.623431 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w" event={"ID":"4bc92a97-0d88-4a83-8e19-a95b7d74759e","Type":"ContainerDied","Data":"2763cd957ec2e5c6977ba0d3ed24ee0974c60ecb58e98e359f8b63e054d710f7"} Jul 12 22:29:02.623488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.623454 2575 scope.go:117] "RemoveContainer" containerID="3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1" Jul 12 22:29:02.633497 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.633478 2575 scope.go:117] "RemoveContainer" containerID="2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4" Jul 12 22:29:02.643284 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.643258 2575 scope.go:117] "RemoveContainer" containerID="3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1" Jul 12 22:29:02.643573 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:29:02.643556 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1\": container with ID starting with 3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1 not found: ID does not exist" containerID="3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1" Jul 12 22:29:02.643634 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.643583 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1"} err="failed to get container status \"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1\": rpc error: code = NotFound desc = could not find container \"3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1\": container with ID starting with 3d24a4113f011c13d73e2adc025e150157a1ab4943fa8bf0576026594c96f3d1 not found: ID does not exist" Jul 12 22:29:02.643634 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.643600 2575 scope.go:117] "RemoveContainer" containerID="2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4" Jul 12 22:29:02.643828 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:29:02.643811 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4\": container with ID starting with 2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4 not found: ID does not exist" containerID="2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4" Jul 12 22:29:02.643881 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.643831 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4"} err="failed to get container status \"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4\": rpc error: code = NotFound desc = could not find container \"2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4\": container with ID starting with 2af838726212f53069e94d07651579272a01d11696a84447b686f83ba777f5a4 not found: ID does not exist" Jul 12 22:29:02.647741 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.647714 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:29:02.652190 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:02.652161 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d3e48a9a-openshift-default-5cbf49646c-7hm6w"] Jul 12 22:29:03.788995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:03.788963 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" path="/var/lib/kubelet/pods/4bc92a97-0d88-4a83-8e19-a95b7d74759e/volumes" Jul 12 22:29:50.261467 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261430 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:29:50.261914 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261872 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.261914 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261883 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.261914 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261900 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.261914 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261907 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.262059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261987 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.262059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.261999 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb25726-e353-45ff-8393-f280bb90c4e2" containerName="istio-proxy" Jul 12 22:29:50.262059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.262013 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc92a97-0d88-4a83-8e19-a95b7d74759e" containerName="istio-proxy" Jul 12 22:29:50.264595 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.264572 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.267965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.267939 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-65f13b-openshift-default-dockercfg-4hjq7\"" Jul 12 22:29:50.276997 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.276965 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:29:50.373861 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.373810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.373861 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.373855 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.373887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjdxf\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.373943 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.373961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.374013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.374042 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374073 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.374074 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.374319 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.374132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475124 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475124 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475154 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475630 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475630 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475630 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475487 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjdxf\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475794 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475756 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475852 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475795 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.475852 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.475807 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.476041 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.476014 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.476129 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.476108 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.477706 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.477684 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.477946 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.477925 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.485035 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.484988 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjdxf\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.485465 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.485441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token\") pod \"e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.576288 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.576183 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:50.733242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.733199 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:29:50.736068 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:29:50.736032 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27d8e79_cf0a_406d_8b90_4a0676319080.slice/crio-e4622afa6b033046b262d6eb946d7876e2efdbcc935a291665a174b7b576f41d WatchSource:0}: Error finding container e4622afa6b033046b262d6eb946d7876e2efdbcc935a291665a174b7b576f41d: Status 404 returned error can't find the container with id e4622afa6b033046b262d6eb946d7876e2efdbcc935a291665a174b7b576f41d Jul 12 22:29:50.738304 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.738275 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:50.738376 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.738341 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:50.738424 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.738399 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:50.879532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.879415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerStarted","Data":"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245"} Jul 12 22:29:50.879532 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.879478 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerStarted","Data":"e4622afa6b033046b262d6eb946d7876e2efdbcc935a291665a174b7b576f41d"} Jul 12 22:29:50.900997 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:50.900940 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podStartSLOduration=0.900920944 podStartE2EDuration="900.920944ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:29:50.89981672 +0000 UTC m=+2239.597304301" watchObservedRunningTime="2026-07-12 22:29:50.900920944 +0000 UTC m=+2239.598408524" Jul 12 22:29:51.576781 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:51.576736 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:51.583357 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:51.583324 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:51.886301 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:51.886268 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:51.887388 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:51.887367 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:29:57.903042 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.903002 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:29:57.906289 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.906262 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.908916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.908891 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6976fc-openshift-default-dockercfg-gzvss\"" Jul 12 22:29:57.920113 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.920084 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:29:57.949690 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949650 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.949867 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949692 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.949867 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949734 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.949867 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmd4s\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.949977 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949866 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.949977 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949893 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.950041 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.949980 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.950041 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.950005 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:57.950178 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:57.950061 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.051626 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.051626 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051628 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.051880 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051668 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.051880 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmd4s\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.051880 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052050 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051875 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052050 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.051994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052050 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052066 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052118 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052440 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.052887 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.052862 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.054353 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.054325 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.054585 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.054561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.060601 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.060569 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.060726 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.060623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmd4s\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s\") pod \"e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.219778 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.219675 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:58.384147 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.384117 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:29:58.386755 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:29:58.386712 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d74228_d9fb_4fc7_ad59_721cedc1e2fc.slice/crio-d6eda5895935442816134edf7ffefb759267ce7cefc8c8e66c73e39277ff3c80 WatchSource:0}: Error finding container d6eda5895935442816134edf7ffefb759267ce7cefc8c8e66c73e39277ff3c80: Status 404 returned error can't find the container with id d6eda5895935442816134edf7ffefb759267ce7cefc8c8e66c73e39277ff3c80 Jul 12 22:29:58.389256 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.389187 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:58.389391 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.389280 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:58.389391 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.389335 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:29:58.930221 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.930161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerStarted","Data":"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267"} Jul 12 22:29:58.930586 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.930240 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerStarted","Data":"d6eda5895935442816134edf7ffefb759267ce7cefc8c8e66c73e39277ff3c80"} Jul 12 22:29:58.951944 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:58.951885 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podStartSLOduration=1.951867164 podStartE2EDuration="1.951867164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:29:58.950302193 +0000 UTC m=+2247.647789774" watchObservedRunningTime="2026-07-12 22:29:58.951867164 +0000 UTC m=+2247.649354744" Jul 12 22:29:59.220107 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:59.220016 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:59.224794 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:59.224767 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:59.936548 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:59.936520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:29:59.937639 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:29:59.937620 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:30:07.928445 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:07.928393 2575 patch_prober.go:28] interesting pod/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 22:30:07.928946 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:07.928489 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 22:30:09.996930 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.996893 2575 generic.go:356] "Generic (PLEG): container finished" podID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerID="4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245" exitCode=0 Jul 12 22:30:09.997392 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.996965 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerDied","Data":"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245"} Jul 12 22:30:09.997441 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.997407 2575 scope.go:117] "RemoveContainer" containerID="4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245" Jul 12 22:30:09.998513 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.998485 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:09.998580 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.998558 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:09.998618 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:09.998585 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:11.012342 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:11.012279 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerStarted","Data":"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf"} Jul 12 22:30:11.576406 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:11.576368 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:30:11.576662 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:11.576632 2575 patch_prober.go:28] interesting pod/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 12 22:30:11.576746 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:11.576698 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 12 22:30:12.577536 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:12.577495 2575 patch_prober.go:28] interesting pod/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 12 22:30:12.577927 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:12.577562 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 12 22:30:13.577359 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:13.577326 2575 patch_prober.go:28] interesting pod/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 12 22:30:13.577617 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:13.577593 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 12 22:30:14.577156 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:14.577111 2575 patch_prober.go:28] interesting pod/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 12 22:30:14.577387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:14.577179 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 12 22:30:15.584904 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:15.584789 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:30:15.585785 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:15.585755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:30:15.586850 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:15.586832 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:30:19.070383 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.070347 2575 generic.go:356] "Generic (PLEG): container finished" podID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerID="857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267" exitCode=0 Jul 12 22:30:19.070821 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.070421 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerDied","Data":"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267"} Jul 12 22:30:19.070880 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.070865 2575 scope.go:117] "RemoveContainer" containerID="857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267" Jul 12 22:30:19.072067 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.072039 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:19.072147 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.072124 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:19.072190 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:19.072167 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:30:20.078079 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:20.078048 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerStarted","Data":"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301"} Jul 12 22:30:20.220375 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:20.220334 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:30:20.220577 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:20.220555 2575 patch_prober.go:28] interesting pod/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 12 22:30:20.220619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:20.220603 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 12 22:30:21.220812 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:21.220770 2575 patch_prober.go:28] interesting pod/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 12 22:30:21.221327 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:21.220832 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 12 22:30:22.220247 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:22.220189 2575 patch_prober.go:28] interesting pod/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 12 22:30:22.220493 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:22.220282 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 12 22:30:23.221134 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:23.221088 2575 patch_prober.go:28] interesting pod/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 12 22:30:23.221580 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:23.221168 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 12 22:30:24.224236 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:24.224188 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:30:24.224619 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:24.224484 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:30:24.225171 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:30:24.225150 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:31:35.937577 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:35.937540 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:31:35.938104 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:35.937756 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" containerID="cri-o://43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf" gracePeriod=30 Jul 12 22:31:41.205431 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.205354 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:31:41.226541 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226505 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226541 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226541 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226800 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226598 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226800 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226628 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226800 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226655 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226800 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226778 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjdxf\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226851 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226858 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:31:41.226996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226903 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.226996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226875 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:31:41.226996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.226960 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data\") pod \"f27d8e79-cf0a-406d-8b90-4a0676319080\" (UID: \"f27d8e79-cf0a-406d-8b90-4a0676319080\") " Jul 12 22:31:41.227363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227121 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:31:41.227363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227308 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:31:41.227477 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227391 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.227477 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227411 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.227477 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227425 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.227477 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227437 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27d8e79-cf0a-406d-8b90-4a0676319080-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.227995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.227967 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data" (OuterVolumeSpecName: "istio-data") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:31:41.229051 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.229019 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:31:41.229146 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.229047 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token" (OuterVolumeSpecName: "istio-token") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:31:41.229146 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.229107 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:31:41.229146 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.229116 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf" (OuterVolumeSpecName: "kube-api-access-zjdxf") pod "f27d8e79-cf0a-406d-8b90-4a0676319080" (UID: "f27d8e79-cf0a-406d-8b90-4a0676319080"). InnerVolumeSpecName "kube-api-access-zjdxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:31:41.327993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.327950 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.327993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.327983 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.327993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.327992 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.327993 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.328000 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjdxf\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-kube-api-access-zjdxf\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.328293 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.328012 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27d8e79-cf0a-406d-8b90-4a0676319080-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:31:41.529775 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.529669 2575 generic.go:356] "Generic (PLEG): container finished" podID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerID="43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf" exitCode=0 Jul 12 22:31:41.529775 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.529718 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" Jul 12 22:31:41.529775 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.529758 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerDied","Data":"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf"} Jul 12 22:31:41.530004 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.529811 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g" event={"ID":"f27d8e79-cf0a-406d-8b90-4a0676319080","Type":"ContainerDied","Data":"e4622afa6b033046b262d6eb946d7876e2efdbcc935a291665a174b7b576f41d"} Jul 12 22:31:41.530004 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.529840 2575 scope.go:117] "RemoveContainer" containerID="43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf" Jul 12 22:31:41.539934 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.539915 2575 scope.go:117] "RemoveContainer" containerID="4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245" Jul 12 22:31:41.548793 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.548771 2575 scope.go:117] "RemoveContainer" containerID="43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf" Jul 12 22:31:41.549077 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:31:41.549058 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf\": container with ID starting with 43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf not found: ID does not exist" containerID="43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf" Jul 12 22:31:41.549131 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.549084 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf"} err="failed to get container status \"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf\": rpc error: code = NotFound desc = could not find container \"43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf\": container with ID starting with 43ed1fd1796e1d86809aabe91c7dfb302d21583a8e7bb60df3af7ef6ec4a2dcf not found: ID does not exist" Jul 12 22:31:41.549131 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.549102 2575 scope.go:117] "RemoveContainer" containerID="4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245" Jul 12 22:31:41.549401 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:31:41.549384 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245\": container with ID starting with 4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245 not found: ID does not exist" containerID="4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245" Jul 12 22:31:41.549451 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.549405 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245"} err="failed to get container status \"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245\": rpc error: code = NotFound desc = could not find container \"4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245\": container with ID starting with 4b26727181f50caaf69f406c45479f97d85535d203eb65e7911b17e419ca3245 not found: ID does not exist" Jul 12 22:31:41.553516 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.553490 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:31:41.558367 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.558343 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-65f13b-openshift-default-698f745cf9-jrg4g"] Jul 12 22:31:41.789040 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:31:41.788950 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" path="/var/lib/kubelet/pods/f27d8e79-cf0a-406d-8b90-4a0676319080/volumes" Jul 12 22:32:02.122401 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:02.122362 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:32:02.122952 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:02.122632 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" containerID="cri-o://2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301" gracePeriod=30 Jul 12 22:32:07.393375 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.393298 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:32:07.481258 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481195 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481258 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481259 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481519 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481290 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481519 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481382 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481519 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481434 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481519 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481458 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmd4s\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481735 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481529 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481735 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481560 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481735 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481563 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:32:07.481735 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481603 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket\") pod \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\" (UID: \"62d74228-d9fb-4fc7-ad59-721cedc1e2fc\") " Jul 12 22:32:07.481945 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481841 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:32:07.481945 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.481861 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.482112 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.482091 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:32:07.482451 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.482415 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:32:07.483004 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.482974 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:32:07.484017 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.483985 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s" (OuterVolumeSpecName: "kube-api-access-kmd4s") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "kube-api-access-kmd4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:32:07.484152 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.484129 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:32:07.484380 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.484356 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:32:07.484572 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.484552 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "62d74228-d9fb-4fc7-ad59-721cedc1e2fc" (UID: "62d74228-d9fb-4fc7-ad59-721cedc1e2fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:32:07.582561 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582515 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582561 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582558 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmd4s\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-kube-api-access-kmd4s\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582576 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582590 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582604 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582617 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582632 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.582770 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.582644 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62d74228-d9fb-4fc7-ad59-721cedc1e2fc-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:32:07.676016 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.675931 2575 generic.go:356] "Generic (PLEG): container finished" podID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerID="2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301" exitCode=0 Jul 12 22:32:07.676016 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.675984 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" Jul 12 22:32:07.676252 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.676017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerDied","Data":"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301"} Jul 12 22:32:07.676252 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.676071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr" event={"ID":"62d74228-d9fb-4fc7-ad59-721cedc1e2fc","Type":"ContainerDied","Data":"d6eda5895935442816134edf7ffefb759267ce7cefc8c8e66c73e39277ff3c80"} Jul 12 22:32:07.676252 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.676096 2575 scope.go:117] "RemoveContainer" containerID="2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301" Jul 12 22:32:07.686059 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.685987 2575 scope.go:117] "RemoveContainer" containerID="857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267" Jul 12 22:32:07.695309 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.695288 2575 scope.go:117] "RemoveContainer" containerID="2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301" Jul 12 22:32:07.695619 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:32:07.695600 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301\": container with ID starting with 2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301 not found: ID does not exist" containerID="2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301" Jul 12 22:32:07.695660 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.695632 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301"} err="failed to get container status \"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301\": rpc error: code = NotFound desc = could not find container \"2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301\": container with ID starting with 2d7e4784b78fecd9cfbc1d95aa88f278e8471bcd977809a2fc12e46347c78301 not found: ID does not exist" Jul 12 22:32:07.695660 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.695654 2575 scope.go:117] "RemoveContainer" containerID="857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267" Jul 12 22:32:07.695963 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:32:07.695942 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267\": container with ID starting with 857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267 not found: ID does not exist" containerID="857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267" Jul 12 22:32:07.696015 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.695969 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267"} err="failed to get container status \"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267\": rpc error: code = NotFound desc = could not find container \"857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267\": container with ID starting with 857ffba09dee4219ea11b766e067d8c2b6c8ec84b5dbbbadc71a3d12a13cb267 not found: ID does not exist" Jul 12 22:32:07.702307 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.702175 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:32:07.705259 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.705200 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6976fc-openshift-default-757b977fb9-qdzqr"] Jul 12 22:32:07.788858 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:07.788822 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" path="/var/lib/kubelet/pods/62d74228-d9fb-4fc7-ad59-721cedc1e2fc/volumes" Jul 12 22:32:13.573031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.572994 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:32:13.573474 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573458 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.573474 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573475 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573493 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573499 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573517 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573523 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573530 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:32:13.573555 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573535 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:32:13.573743 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573593 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:32:13.573743 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573603 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.573743 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.573613 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d74228-d9fb-4fc7-ad59-721cedc1e2fc" containerName="istio-proxy" Jul 12 22:32:13.576929 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.576905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.581551 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.581486 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7dcad0-openshift-default-dockercfg-9ww5r\"" Jul 12 22:32:13.599572 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.599533 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:32:13.744962 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.744919 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.744962 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.744959 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.744996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745087 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8bl8\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745200 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745194 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745430 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.745430 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.745291 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846639 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846534 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846639 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8bl8\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846639 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846615 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846947 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846947 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.846947 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846856 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.846996 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847061 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847141 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.847490 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.848020 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.847996 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.848977 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.848958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.849422 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.849406 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.857439 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.857400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.857603 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.857552 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8bl8\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8\") pod \"e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:13.899509 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:13.899463 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:14.062631 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.062601 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:32:14.071285 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.071244 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:14.071401 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.071336 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:14.071401 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.071381 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:14.723964 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.723927 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerStarted","Data":"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515"} Jul 12 22:32:14.724366 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.723995 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerStarted","Data":"ee1c6c633166b626f6c535828f72d05095895826d58c4b15b1664ebe69004ca5"} Jul 12 22:32:14.746255 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.746178 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podStartSLOduration=1.7461594950000001 podStartE2EDuration="1.746159495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:32:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:32:14.743355496 +0000 UTC m=+2383.440843087" watchObservedRunningTime="2026-07-12 22:32:14.746159495 +0000 UTC m=+2383.443647075" Jul 12 22:32:14.899740 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:14.899702 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:15.906025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:15.905995 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:16.735831 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:16.735794 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:16.737271 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:16.737241 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:28.825270 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.825235 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerID="828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515" exitCode=0 Jul 12 22:32:28.825800 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.825303 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerDied","Data":"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515"} Jul 12 22:32:28.825882 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.825861 2575 scope.go:117] "RemoveContainer" containerID="828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515" Jul 12 22:32:28.827062 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.827035 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:28.827181 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.827121 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:28.827181 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:28.827168 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:32:29.833012 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:29.832974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerStarted","Data":"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6"} Jul 12 22:32:29.900972 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:29.900929 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:29.901937 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:29.901898 2575 patch_prober.go:28] interesting pod/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 12 22:32:29.902083 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:29.901966 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 12 22:32:30.900792 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:30.900743 2575 patch_prober.go:28] interesting pod/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 12 22:32:30.901342 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:30.900814 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 12 22:32:31.735864 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:31.735825 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:31.900668 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:31.900618 2575 patch_prober.go:28] interesting pod/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 12 22:32:31.900882 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:31.900692 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 12 22:32:32.900243 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:32.900184 2575 patch_prober.go:28] interesting pod/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 12 22:32:32.900598 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:32.900567 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 12 22:32:33.904275 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:33.904244 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:32:33.905262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:32:33.905242 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:33:20.865965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:20.865923 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:33:20.867064 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:20.867007 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" containerID="cri-o://0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6" gracePeriod=30 Jul 12 22:33:26.139428 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.139344 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:33:26.142501 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.142473 2575 generic.go:356] "Generic (PLEG): container finished" podID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerID="0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6" exitCode=0 Jul 12 22:33:26.142642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.142518 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" Jul 12 22:33:26.142642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.142548 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerDied","Data":"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6"} Jul 12 22:33:26.142642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.142609 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td" event={"ID":"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96","Type":"ContainerDied","Data":"ee1c6c633166b626f6c535828f72d05095895826d58c4b15b1664ebe69004ca5"} Jul 12 22:33:26.142642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.142643 2575 scope.go:117] "RemoveContainer" containerID="0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6" Jul 12 22:33:26.152125 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.152100 2575 scope.go:117] "RemoveContainer" containerID="828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515" Jul 12 22:33:26.161869 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.161778 2575 scope.go:117] "RemoveContainer" containerID="0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6" Jul 12 22:33:26.162226 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:33:26.162179 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6\": container with ID starting with 0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6 not found: ID does not exist" containerID="0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6" Jul 12 22:33:26.162346 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.162240 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6"} err="failed to get container status \"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6\": rpc error: code = NotFound desc = could not find container \"0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6\": container with ID starting with 0fa2ee2394eb77913198d64538445bb6a59db8e990b2299a5ba41be9376479f6 not found: ID does not exist" Jul 12 22:33:26.162346 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.162276 2575 scope.go:117] "RemoveContainer" containerID="828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515" Jul 12 22:33:26.162639 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:33:26.162612 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515\": container with ID starting with 828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515 not found: ID does not exist" containerID="828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515" Jul 12 22:33:26.162725 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.162644 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515"} err="failed to get container status \"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515\": rpc error: code = NotFound desc = could not find container \"828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515\": container with ID starting with 828f3ba9a980a319623f01ac98d3bbeb986584612fcc673852e5ec5da07b4515 not found: ID does not exist" Jul 12 22:33:26.279566 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279528 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.279784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279589 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8bl8\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.279784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279620 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.279784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279659 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.279784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279733 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.279784 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279760 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.280081 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279798 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.280081 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279843 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.280081 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.279885 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket\") pod \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\" (UID: \"d9a119ab-72c7-4fcd-94d7-3dd5ca976a96\") " Jul 12 22:33:26.280699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.280536 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:33:26.280699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.280560 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:33:26.284008 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.283963 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:33:26.284148 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.284057 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:33:26.284244 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.284224 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:33:26.284710 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.284683 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:33:26.285177 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.285147 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:33:26.285437 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.285403 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:33:26.285912 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.285882 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8" (OuterVolumeSpecName: "kube-api-access-x8bl8") pod "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" (UID: "d9a119ab-72c7-4fcd-94d7-3dd5ca976a96"). InnerVolumeSpecName "kube-api-access-x8bl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:33:26.381947 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.381894 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8bl8\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-kube-api-access-x8bl8\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.381947 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.381944 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.381963 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.381975 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.381989 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.382001 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.382013 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.382025 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.382242 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.382038 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:33:26.469527 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.469488 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:33:26.476965 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:26.476938 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7dcad0-openshift-default-76cbd8b5ff-wx9td"] Jul 12 22:33:27.789020 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:27.788979 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" path="/var/lib/kubelet/pods/d9a119ab-72c7-4fcd-94d7-3dd5ca976a96/volumes" Jul 12 22:33:29.866879 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.866839 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:33:29.867387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867371 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.867438 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867391 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.867438 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867406 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.867438 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867412 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.867534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867491 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.867534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867499 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27d8e79-cf0a-406d-8b90-4a0676319080" containerName="istio-proxy" Jul 12 22:33:29.867534 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.867507 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a119ab-72c7-4fcd-94d7-3dd5ca976a96" containerName="istio-proxy" Jul 12 22:33:29.870856 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.870836 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:29.874304 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.874277 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7a7f11-openshift-default-dockercfg-5bplz\"" Jul 12 22:33:29.890677 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:29.890633 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:33:30.018523 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018480 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5wfs\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.018733 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.019056 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.019056 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.018877 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120419 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120321 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120419 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120478 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120495 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120539 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5wfs\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.120657 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.121048 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120870 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.121048 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.120914 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.121463 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.121199 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.121463 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.121350 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.121463 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.121391 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.124473 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.124446 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.124473 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.124460 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.129402 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.129363 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.129714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.129691 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5wfs\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs\") pod \"e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.182317 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.182277 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:30.335488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.335459 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:33:30.337530 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:33:30.337498 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec5b9f6_d94b_4d6e_a6a0_db6d5fef64dc.slice/crio-75f466f4128621160bbf18228236d39610ddf24ab5d412b13ceb73b81f9ed9e5 WatchSource:0}: Error finding container 75f466f4128621160bbf18228236d39610ddf24ab5d412b13ceb73b81f9ed9e5: Status 404 returned error can't find the container with id 75f466f4128621160bbf18228236d39610ddf24ab5d412b13ceb73b81f9ed9e5 Jul 12 22:33:30.339917 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.339879 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:30.339995 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.339980 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:30.340043 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:30.340025 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:31.184642 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:31.184597 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerStarted","Data":"e099a71579ff2a846d5e61d8e429eb6aab987218632c405df5e5d7acc1eec70e"} Jul 12 22:33:31.185035 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:31.184704 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerStarted","Data":"75f466f4128621160bbf18228236d39610ddf24ab5d412b13ceb73b81f9ed9e5"} Jul 12 22:33:31.211884 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:31.211830 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podStartSLOduration=2.211812728 podStartE2EDuration="2.211812728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:33:31.211158597 +0000 UTC m=+2459.908646177" watchObservedRunningTime="2026-07-12 22:33:31.211812728 +0000 UTC m=+2459.909300308" Jul 12 22:33:32.182591 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:32.182552 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:32.188531 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:32.188500 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:32.191036 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:32.191010 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:32.192013 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:32.191994 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:45.266102 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.266064 2575 generic.go:356] "Generic (PLEG): container finished" podID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerID="e099a71579ff2a846d5e61d8e429eb6aab987218632c405df5e5d7acc1eec70e" exitCode=0 Jul 12 22:33:45.266554 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.266140 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerDied","Data":"e099a71579ff2a846d5e61d8e429eb6aab987218632c405df5e5d7acc1eec70e"} Jul 12 22:33:45.266615 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.266601 2575 scope.go:117] "RemoveContainer" containerID="e099a71579ff2a846d5e61d8e429eb6aab987218632c405df5e5d7acc1eec70e" Jul 12 22:33:45.267638 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.267611 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:45.267700 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.267679 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:45.267736 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:45.267713 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 22:33:46.274961 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:46.274919 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerStarted","Data":"7c955865636aa15798a7fcd90007c935c8ec8f676ffbec6caf002b19d12d0347"} Jul 12 22:33:47.184979 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:47.184941 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:47.185466 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:47.185434 2575 patch_prober.go:28] interesting pod/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 12 22:33:47.185698 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:47.185666 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 12 22:33:47.191477 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:47.191441 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:48.183590 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:48.183548 2575 patch_prober.go:28] interesting pod/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 12 22:33:48.184196 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:48.184160 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 12 22:33:49.183578 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:49.183536 2575 patch_prober.go:28] interesting pod/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 12 22:33:49.183884 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:49.183849 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 12 22:33:50.222824 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:50.222798 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:33:50.224108 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:33:50.224087 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:37:30.641022 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.640980 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:37:30.643889 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.643867 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.648177 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.648154 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-66fe58\"/\"kube-root-ca.crt\"" Jul 12 22:37:30.648352 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.648154 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-66fe58\"/\"openshift-service-ca.crt\"" Jul 12 22:37:30.648352 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.648266 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-66fe58\"/\"sub-test-model-66fe58-kserve-self-signed-certs\"" Jul 12 22:37:30.648352 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.648193 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-66fe58\"/\"default-dockercfg-c4gbw\"" Jul 12 22:37:30.654531 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.654502 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:37:30.727870 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.727828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.728078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.727900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.728078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.727945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.728078 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.728021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.728282 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.728082 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zjjv\" (UniqueName: \"kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.728282 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.728130 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829513 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829468 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829535 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829581 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zjjv\" (UniqueName: \"kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829653 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.829932 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829722 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.830004 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.829981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.830064 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.830006 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.830124 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.830055 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.832185 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.832156 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.832463 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.832447 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.839759 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.839729 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zjjv\" (UniqueName: \"kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv\") pod \"sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:30.954573 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:30.954469 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:31.110144 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:31.110114 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:37:31.113124 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:37:31.113086 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a57daae_3796_4a1e_b98e_f0c0786a3f2a.slice/crio-b56d52cab3bcb7d6eab4b5caa207171ce8a1a624de61cddaf017ca1a074f6fd9 WatchSource:0}: Error finding container b56d52cab3bcb7d6eab4b5caa207171ce8a1a624de61cddaf017ca1a074f6fd9: Status 404 returned error can't find the container with id b56d52cab3bcb7d6eab4b5caa207171ce8a1a624de61cddaf017ca1a074f6fd9 Jul 12 22:37:31.516801 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:31.516763 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" event={"ID":"7a57daae-3796-4a1e-b98e-f0c0786a3f2a","Type":"ContainerStarted","Data":"3cef1b9640bafb42f879ce39a5b99e9ba047e73bdbfec685b874bf4a2c4d2c93"} Jul 12 22:37:31.517023 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:31.516811 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" event={"ID":"7a57daae-3796-4a1e-b98e-f0c0786a3f2a","Type":"ContainerStarted","Data":"b56d52cab3bcb7d6eab4b5caa207171ce8a1a624de61cddaf017ca1a074f6fd9"} Jul 12 22:37:31.535374 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:31.535316 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" podStartSLOduration=1.5352980999999999 podStartE2EDuration="1.5352981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 22:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:37:31.534979916 +0000 UTC m=+2700.232467508" watchObservedRunningTime="2026-07-12 22:37:31.5352981 +0000 UTC m=+2700.232785679" Jul 12 22:37:40.955342 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:40.955300 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:40.955782 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:40.955365 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:40.968563 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:40.968535 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:37:41.582867 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:37:41.582832 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:39:02.857422 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:02.857319 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:39:17.879037 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:17.878969 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" podUID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" containerName="main" containerID="cri-o://3cef1b9640bafb42f879ce39a5b99e9ba047e73bdbfec685b874bf4a2c4d2c93" gracePeriod=45 Jul 12 22:39:18.129248 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.129148 2575 generic.go:356] "Generic (PLEG): container finished" podID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" containerID="3cef1b9640bafb42f879ce39a5b99e9ba047e73bdbfec685b874bf4a2c4d2c93" exitCode=0 Jul 12 22:39:18.129414 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.129243 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" event={"ID":"7a57daae-3796-4a1e-b98e-f0c0786a3f2a","Type":"ContainerDied","Data":"3cef1b9640bafb42f879ce39a5b99e9ba047e73bdbfec685b874bf4a2c4d2c93"} Jul 12 22:39:18.167424 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.167392 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:39:18.234226 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234169 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.234403 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234248 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.234403 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.234500 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234469 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zjjv\" (UniqueName: \"kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.234622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234500 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home" (OuterVolumeSpecName: "home") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:39:18.234622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234529 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:39:18.234622 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234564 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.234906 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234629 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache\") pod \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\" (UID: \"7a57daae-3796-4a1e-b98e-f0c0786a3f2a\") " Jul 12 22:39:18.235025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234946 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:39:18.235025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234976 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-home\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:18.235025 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.234994 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tmp-dir\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:18.236844 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.236810 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm" (OuterVolumeSpecName: "dshm") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:39:18.237011 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.236973 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 22:39:18.237011 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.236975 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv" (OuterVolumeSpecName: "kube-api-access-2zjjv") pod "7a57daae-3796-4a1e-b98e-f0c0786a3f2a" (UID: "7a57daae-3796-4a1e-b98e-f0c0786a3f2a"). InnerVolumeSpecName "kube-api-access-2zjjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:39:18.335637 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.335596 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-tls-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:18.335637 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.335628 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zjjv\" (UniqueName: \"kubernetes.io/projected/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-kube-api-access-2zjjv\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:18.335637 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.335639 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-dshm\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:18.335637 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:18.335649 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a57daae-3796-4a1e-b98e-f0c0786a3f2a-model-cache\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:39:19.135608 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.135578 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" Jul 12 22:39:19.136031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.135617 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh" event={"ID":"7a57daae-3796-4a1e-b98e-f0c0786a3f2a","Type":"ContainerDied","Data":"b56d52cab3bcb7d6eab4b5caa207171ce8a1a624de61cddaf017ca1a074f6fd9"} Jul 12 22:39:19.136031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.135676 2575 scope.go:117] "RemoveContainer" containerID="3cef1b9640bafb42f879ce39a5b99e9ba047e73bdbfec685b874bf4a2c4d2c93" Jul 12 22:39:19.159057 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.159021 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:39:19.162679 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.162651 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-66fe58/sub-test-model-66fe58-kserve-5b8d9bcf5f-hlzsh"] Jul 12 22:39:19.789503 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:39:19.789460 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" path="/var/lib/kubelet/pods/7a57daae-3796-4a1e-b98e-f0c0786a3f2a/volumes" Jul 12 22:40:32.813117 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.813070 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7553-140033590961184"] Jul 12 22:40:32.813754 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.813733 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" containerName="main" Jul 12 22:40:32.813846 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.813757 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" containerName="main" Jul 12 22:40:32.813901 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.813892 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a57daae-3796-4a1e-b98e-f0c0786a3f2a" containerName="main" Jul 12 22:40:32.816128 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.816107 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:32.818699 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.818681 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-bs2xj\"" Jul 12 22:40:32.822963 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.822940 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7553-140033590961184"] Jul 12 22:40:32.992377 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:32.992336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c87m\" (UniqueName: \"kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m\") pod \"test-curl-7553-140033590961184\" (UID: \"3e5336b9-c3e7-4b23-ba1a-2baa0573d180\") " pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:33.093131 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.093034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c87m\" (UniqueName: \"kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m\") pod \"test-curl-7553-140033590961184\" (UID: \"3e5336b9-c3e7-4b23-ba1a-2baa0573d180\") " pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:33.101822 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.101789 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c87m\" (UniqueName: \"kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m\") pod \"test-curl-7553-140033590961184\" (UID: \"3e5336b9-c3e7-4b23-ba1a-2baa0573d180\") " pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:33.126855 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.126818 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:33.269612 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.269568 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7553-140033590961184"] Jul 12 22:40:33.271930 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:40:33.271897 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e5336b9_c3e7_4b23_ba1a_2baa0573d180.slice/crio-57898ef17222d9311b345d2268e6f8d1beb2936434363cf12bc62230e9662930 WatchSource:0}: Error finding container 57898ef17222d9311b345d2268e6f8d1beb2936434363cf12bc62230e9662930: Status 404 returned error can't find the container with id 57898ef17222d9311b345d2268e6f8d1beb2936434363cf12bc62230e9662930 Jul 12 22:40:33.273262 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.273244 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 22:40:33.552293 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:33.552255 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7553-140033590961184" event={"ID":"3e5336b9-c3e7-4b23-ba1a-2baa0573d180","Type":"ContainerStarted","Data":"57898ef17222d9311b345d2268e6f8d1beb2936434363cf12bc62230e9662930"} Jul 12 22:40:34.559178 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:34.559143 2575 generic.go:356] "Generic (PLEG): container finished" podID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" containerID="786691d0365e62e21530ce405bf48714b597d8369cba75c387bf86ab11f7bcc4" exitCode=0 Jul 12 22:40:34.559579 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:34.559197 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7553-140033590961184" event={"ID":"3e5336b9-c3e7-4b23-ba1a-2baa0573d180","Type":"ContainerDied","Data":"786691d0365e62e21530ce405bf48714b597d8369cba75c387bf86ab11f7bcc4"} Jul 12 22:40:35.721517 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.721483 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:35.760821 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.760785 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7553-140033590961184_3e5336b9-c3e7-4b23-ba1a-2baa0573d180/test-curl-7553-140033590961184/0.log" Jul 12 22:40:35.791341 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.791309 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7553-140033590961184"] Jul 12 22:40:35.791501 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.791354 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7553-140033590961184"] Jul 12 22:40:35.818816 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.818730 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c87m\" (UniqueName: \"kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m\") pod \"3e5336b9-c3e7-4b23-ba1a-2baa0573d180\" (UID: \"3e5336b9-c3e7-4b23-ba1a-2baa0573d180\") " Jul 12 22:40:35.820855 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.820830 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m" (OuterVolumeSpecName: "kube-api-access-4c87m") pod "3e5336b9-c3e7-4b23-ba1a-2baa0573d180" (UID: "3e5336b9-c3e7-4b23-ba1a-2baa0573d180"). InnerVolumeSpecName "kube-api-access-4c87m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:40:35.920154 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:35.920116 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c87m\" (UniqueName: \"kubernetes.io/projected/3e5336b9-c3e7-4b23-ba1a-2baa0573d180-kube-api-access-4c87m\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:40:36.570734 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:36.570708 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57898ef17222d9311b345d2268e6f8d1beb2936434363cf12bc62230e9662930" Jul 12 22:40:36.570734 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:36.570726 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:36.581682 ip-10-0-132-6 kubenswrapper[2575]: E0712 22:40:36.581652 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7553-140033590961184\" is forbidden: User \"system:node:ip-10-0-132-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-6.ec2.internal' and this object" podUID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" pod="openshift-ingress/test-curl-7553-140033590961184" Jul 12 22:40:37.791308 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:40:37.790187 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" path="/var/lib/kubelet/pods/3e5336b9-c3e7-4b23-ba1a-2baa0573d180/volumes" Jul 12 22:41:30.725991 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:30.725948 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:41:30.726562 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:30.726275 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" containerID="cri-o://7c955865636aa15798a7fcd90007c935c8ec8f676ffbec6caf002b19d12d0347" gracePeriod=30 Jul 12 22:41:31.380952 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:31.380914 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d5d984dc-46t8v_ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71/authorino/0.log" Jul 12 22:41:35.225786 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.225747 2575 patch_prober.go:28] interesting pod/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 22:41:35.226158 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.225821 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 22:41:35.897363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.897327 2575 generic.go:356] "Generic (PLEG): container finished" podID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerID="7c955865636aa15798a7fcd90007c935c8ec8f676ffbec6caf002b19d12d0347" exitCode=0 Jul 12 22:41:35.897560 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.897401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerDied","Data":"7c955865636aa15798a7fcd90007c935c8ec8f676ffbec6caf002b19d12d0347"} Jul 12 22:41:35.897560 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.897480 2575 scope.go:117] "RemoveContainer" containerID="e099a71579ff2a846d5e61d8e429eb6aab987218632c405df5e5d7acc1eec70e" Jul 12 22:41:35.992605 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:35.992528 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:41:36.104520 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.104487 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h8cc7_3936ebea-d272-4f7c-bb93-f2386ec51915/manager/0.log" Jul 12 22:41:36.113941 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.113910 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114034 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.113958 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114034 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.113980 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114110 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114035 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114151 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114128 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114329 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114289 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114293 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:41:36.114481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114308 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:41:36.114481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114400 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5wfs\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114443 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114481 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114477 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket\") pod \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\" (UID: \"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc\") " Jul 12 22:41:36.114756 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114546 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 22:41:36.114815 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114804 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.114874 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114823 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-workload-certs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.114874 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114836 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istiod-ca-cert\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.114976 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114880 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:41:36.115031 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.114976 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:41:36.116488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.116461 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 22:41:36.116488 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.116464 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 22:41:36.116773 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.116745 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs" (OuterVolumeSpecName: "kube-api-access-r5wfs") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "kube-api-access-r5wfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:41:36.116868 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.116769 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" (UID: "3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 22:41:36.215608 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215564 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-data\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.215608 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215598 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-token\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.215608 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215613 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5wfs\" (UniqueName: \"kubernetes.io/projected/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-kube-api-access-r5wfs\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.215876 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215626 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-envoy\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.215876 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215638 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-credential-socket\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.215876 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.215651 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc-istio-podinfo\") on node \"ip-10-0-132-6.ec2.internal\" DevicePath \"\"" Jul 12 22:41:36.224441 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.224405 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7bgfk_f65507a5-3e59-4ee5-bb70-23248e52acb7/manager/0.log" Jul 12 22:41:36.348902 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.348804 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6df67c4d8d-hk9sv_da1ca873-df0e-4f32-a757-dc2784b68972/manager/0.log" Jul 12 22:41:36.461193 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.461163 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vj2hh_b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d/server/0.log" Jul 12 22:41:36.575990 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.575949 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2dhns_5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2/manager/0.log" Jul 12 22:41:36.903702 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.903668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" event={"ID":"3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc","Type":"ContainerDied","Data":"75f466f4128621160bbf18228236d39610ddf24ab5d412b13ceb73b81f9ed9e5"} Jul 12 22:41:36.903702 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.903696 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7" Jul 12 22:41:36.903939 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.903732 2575 scope.go:117] "RemoveContainer" containerID="7c955865636aa15798a7fcd90007c935c8ec8f676ffbec6caf002b19d12d0347" Jul 12 22:41:36.927149 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.927115 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:41:36.932081 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.932055 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d867dd548-rsdbp_bbf0987a-c7be-4b37-9aa7-cb729d6886ba/copy-manifests/0.log" Jul 12 22:41:36.933284 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.933265 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7a7f11-openshift-default-655fbc6d5f-vhwt7"] Jul 12 22:41:36.939768 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:36.939737 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d867dd548-rsdbp_bbf0987a-c7be-4b37-9aa7-cb729d6886ba/manager/0.log" Jul 12 22:41:37.304714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:37.304628 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7845b57fd5-9wk9v_48b17f97-2a30-4537-ab2e-f03b75d68972/maas-api/0.log" Jul 12 22:41:37.788684 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:37.788648 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" path="/var/lib/kubelet/pods/3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc/volumes" Jul 12 22:41:40.171666 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.171636 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d5d984dc-46t8v_ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71/authorino/0.log" Jul 12 22:41:40.288588 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.288558 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8s2sl_e97d212b-ecbc-4c60-a560-bdc7402ec138/manager/0.log" Jul 12 22:41:40.395307 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.395274 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b2gv6_800c07ad-1ada-4351-95dc-b0d8524f617b/manager/0.log" Jul 12 22:41:40.629403 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.629325 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-prsbm_576eae94-fea8-4dec-b1ea-5965ce3c7813/registry-server/0.log" Jul 12 22:41:40.750600 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.750567 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ggwrb_7ebb49d8-1443-4205-a5cb-582db2bf94c7/manager/0.log" Jul 12 22:41:40.982910 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:40.982884 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nwmrw_79e7fe86-9771-4ac3-b3cf-3479f1d08589/manager/0.log" Jul 12 22:41:41.323029 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:41.322942 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c75d_ae0e1df6-a811-41c6-8856-ae0feeeef971/istio-proxy/0.log" Jul 12 22:41:41.544941 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:41.544912 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b84d88d8-s6s7s_dbaf50e2-97f6-49a9-9273-a35f8f3d8c44/kube-auth-proxy/0.log" Jul 12 22:41:41.773196 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:41.773163 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/1.log" Jul 12 22:41:41.895550 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:41.895517 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-262s7_6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f/payload-pre-processing/0.log" Jul 12 22:41:42.016011 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:42.015977 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-8nz58_411c6caf-3243-4906-bdd8-848fa8ce1b66/payload-processing/0.log" Jul 12 22:41:42.463345 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:42.463314 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wvbn4_d3cddaa0-04b1-458c-a5db-2a2bd79e1fe0/main/0.log" Jul 12 22:41:42.794917 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:42.794829 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq4595_1cc13010-4c11-4f85-a231-a27361829891/main/0.log" Jul 12 22:41:50.047951 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:50.047917 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c7l4v_fee60708-57c1-4b5d-97b6-e5ea3ed089c1/global-pull-secret-syncer/0.log" Jul 12 22:41:50.164280 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:50.164249 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nr65r_48fd652c-b3e0-4244-8b32-c80081d686bc/konnectivity-agent/0.log" Jul 12 22:41:50.185636 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:50.185596 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-6.ec2.internal_62bbbba9f59567f90d3b65dc971a3dd1/haproxy/0.log" Jul 12 22:41:56.138338 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.138257 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d5d984dc-46t8v_ce8a7a5c-fcfa-4068-a5bd-af8c761c4e71/authorino/0.log" Jul 12 22:41:56.176628 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.176594 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8s2sl_e97d212b-ecbc-4c60-a560-bdc7402ec138/manager/0.log" Jul 12 22:41:56.217410 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.217383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b2gv6_800c07ad-1ada-4351-95dc-b0d8524f617b/manager/0.log" Jul 12 22:41:56.321748 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.321718 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-prsbm_576eae94-fea8-4dec-b1ea-5965ce3c7813/registry-server/0.log" Jul 12 22:41:56.512318 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.512232 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ggwrb_7ebb49d8-1443-4205-a5cb-582db2bf94c7/manager/0.log" Jul 12 22:41:56.627715 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:56.627680 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nwmrw_79e7fe86-9771-4ac3-b3cf-3479f1d08589/manager/0.log" Jul 12 22:41:58.317645 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.317614 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/alertmanager/0.log" Jul 12 22:41:58.337475 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.337441 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/config-reloader/0.log" Jul 12 22:41:58.355726 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.355696 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/kube-rbac-proxy-web/0.log" Jul 12 22:41:58.374129 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.374086 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/kube-rbac-proxy/0.log" Jul 12 22:41:58.395508 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.395479 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/kube-rbac-proxy-metric/0.log" Jul 12 22:41:58.415535 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.415504 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/prom-label-proxy/0.log" Jul 12 22:41:58.435880 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.435851 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2d6ca531-8f76-44cf-b096-65c71f46e5c4/init-config-reloader/0.log" Jul 12 22:41:58.625271 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.625176 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96m6h_31fce4f4-ead7-401d-b2f7-2e8c867c76ee/node-exporter/0.log" Jul 12 22:41:58.642714 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.642690 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96m6h_31fce4f4-ead7-401d-b2f7-2e8c867c76ee/kube-rbac-proxy/0.log" Jul 12 22:41:58.660996 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:58.660970 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96m6h_31fce4f4-ead7-401d-b2f7-2e8c867c76ee/init-textfile/0.log" Jul 12 22:41:59.086923 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.086894 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-j7x6m_2bfb785b-992f-42fa-8641-22121b69acd2/prometheus-operator-admission-webhook/0.log" Jul 12 22:41:59.111467 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.111440 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54fb6b9d9-g2dcc_0cf10002-9771-456a-a188-7d8177495837/telemeter-client/0.log" Jul 12 22:41:59.132090 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.132061 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54fb6b9d9-g2dcc_0cf10002-9771-456a-a188-7d8177495837/reload/0.log" Jul 12 22:41:59.150363 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.150333 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54fb6b9d9-g2dcc_0cf10002-9771-456a-a188-7d8177495837/kube-rbac-proxy/0.log" Jul 12 22:41:59.179489 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.179459 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/thanos-query/0.log" Jul 12 22:41:59.197637 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.197609 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/kube-rbac-proxy-web/0.log" Jul 12 22:41:59.216099 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.216070 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/kube-rbac-proxy/0.log" Jul 12 22:41:59.233890 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.233859 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/prom-label-proxy/0.log" Jul 12 22:41:59.251923 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.251894 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/kube-rbac-proxy-rules/0.log" Jul 12 22:41:59.270655 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.270627 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85cd65ffbf-zmfj7_a95b2530-6d67-45f1-8a00-7a84e8924bf1/kube-rbac-proxy-metrics/0.log" Jul 12 22:41:59.908841 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.908810 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn"] Jul 12 22:41:59.909259 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909245 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" containerName="test-curl-7553-140033590961184" Jul 12 22:41:59.909259 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909260 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" containerName="test-curl-7553-140033590961184" Jul 12 22:41:59.909351 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909278 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" Jul 12 22:41:59.909351 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909283 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" Jul 12 22:41:59.909351 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909346 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e5336b9-c3e7-4b23-ba1a-2baa0573d180" containerName="test-curl-7553-140033590961184" Jul 12 22:41:59.909451 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909358 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" Jul 12 22:41:59.909451 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.909366 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec5b9f6-d94b-4d6e-a6a0-db6d5fef64dc" containerName="istio-proxy" Jul 12 22:41:59.914033 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.913992 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:41:59.917179 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.917151 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-grwlh\"/\"default-dockercfg-zbdsv\"" Jul 12 22:41:59.918732 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.918704 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-grwlh\"/\"openshift-service-ca.crt\"" Jul 12 22:41:59.918877 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.918760 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-grwlh\"/\"kube-root-ca.crt\"" Jul 12 22:41:59.921170 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.921146 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn"] Jul 12 22:41:59.926306 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.926283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-proc\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:41:59.926422 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.926321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmx8s\" (UniqueName: \"kubernetes.io/projected/5483bcb3-ebcf-4269-8acd-c7db987b6de2-kube-api-access-xmx8s\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:41:59.926471 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.926435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-podres\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:41:59.926505 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.926481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-sys\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:41:59.926545 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:41:59.926515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-lib-modules\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027482 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-proc\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027664 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmx8s\" (UniqueName: \"kubernetes.io/projected/5483bcb3-ebcf-4269-8acd-c7db987b6de2-kube-api-access-xmx8s\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027664 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-podres\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027664 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-proc\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027664 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-sys\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027664 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027661 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-lib-modules\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027851 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027678 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-sys\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027851 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027781 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-podres\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.027851 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.027783 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5483bcb3-ebcf-4269-8acd-c7db987b6de2-lib-modules\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.035683 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.035655 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmx8s\" (UniqueName: \"kubernetes.io/projected/5483bcb3-ebcf-4269-8acd-c7db987b6de2-kube-api-access-xmx8s\") pod \"perf-node-gather-daemonset-kr7sn\" (UID: \"5483bcb3-ebcf-4269-8acd-c7db987b6de2\") " pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.226569 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.226481 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:00.362217 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:00.362179 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn"] Jul 12 22:42:00.364499 ip-10-0-132-6 kubenswrapper[2575]: W0712 22:42:00.364462 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5483bcb3_ebcf_4269_8acd_c7db987b6de2.slice/crio-983d15a9ac8a9c393ce86aa5fb872bf96d6be41f32bf3055663754c7ac73be4c WatchSource:0}: Error finding container 983d15a9ac8a9c393ce86aa5fb872bf96d6be41f32bf3055663754c7ac73be4c: Status 404 returned error can't find the container with id 983d15a9ac8a9c393ce86aa5fb872bf96d6be41f32bf3055663754c7ac73be4c Jul 12 22:42:01.037750 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:01.037706 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" event={"ID":"5483bcb3-ebcf-4269-8acd-c7db987b6de2","Type":"ContainerStarted","Data":"028272d4d4e34388e74a6f5a739f5a75bd50ee9d33626f35462088646dd9d284"} Jul 12 22:42:01.038127 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:01.037787 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" event={"ID":"5483bcb3-ebcf-4269-8acd-c7db987b6de2","Type":"ContainerStarted","Data":"983d15a9ac8a9c393ce86aa5fb872bf96d6be41f32bf3055663754c7ac73be4c"} Jul 12 22:42:01.038127 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:01.037807 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:01.053985 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:01.053939 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" podStartSLOduration=2.053924192 podStartE2EDuration="2.053924192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 22:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 22:42:01.053307152 +0000 UTC m=+2969.750794732" watchObservedRunningTime="2026-07-12 22:42:01.053924192 +0000 UTC m=+2969.751411762" Jul 12 22:42:01.577470 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:01.577443 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-d8vlr_3a74edf0-42f2-4bad-b978-6f91498803cd/download-server/0.log" Jul 12 22:42:03.238959 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:03.238928 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lmw5c_031e3ac8-87ac-4993-a35f-f0f016f1d0c4/dns/0.log" Jul 12 22:42:03.259395 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:03.259367 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lmw5c_031e3ac8-87ac-4993-a35f-f0f016f1d0c4/kube-rbac-proxy/0.log" Jul 12 22:42:03.334482 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:03.334449 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rsbww_1545efb1-e4a2-4ab6-8001-4137a9a081fe/dns-node-resolver/0.log" Jul 12 22:42:03.880361 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:03.880322 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-fd87957d-4h4ld_4cf50fc4-2925-451b-b448-80bbc82992c0/registry/0.log" Jul 12 22:42:03.949387 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:03.949358 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wktg5_4e55c5f9-fffd-4e4d-ad8f-75ba9e975ef9/node-ca/0.log" Jul 12 22:42:04.845049 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:04.845021 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c75d_ae0e1df6-a811-41c6-8856-ae0feeeef971/istio-proxy/0.log" Jul 12 22:42:04.977807 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:04.977780 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b84d88d8-s6s7s_dbaf50e2-97f6-49a9-9273-a35f8f3d8c44/kube-auth-proxy/0.log" Jul 12 22:42:05.029484 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:05.029447 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/0.log" Jul 12 22:42:05.086792 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:05.086759 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/1.log" Jul 12 22:42:05.127067 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:05.126982 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-262s7_6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f/payload-pre-processing/0.log" Jul 12 22:42:05.165539 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:05.165495 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-8nz58_411c6caf-3243-4906-bdd8-848fa8ce1b66/payload-processing/0.log" Jul 12 22:42:05.967189 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:05.967157 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tvsxt_651f2cc2-fa5e-482c-864e-da1f52675d9d/serve-healthcheck-canary/0.log" Jul 12 22:42:06.439608 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:06.439578 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lxsf_6ac3cb4e-fb16-4f25-a377-bb22b67ad865/kube-rbac-proxy/0.log" Jul 12 22:42:06.465386 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:06.465355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lxsf_6ac3cb4e-fb16-4f25-a377-bb22b67ad865/exporter/0.log" Jul 12 22:42:06.487292 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:06.487262 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4lxsf_6ac3cb4e-fb16-4f25-a377-bb22b67ad865/extractor/0.log" Jul 12 22:42:07.053353 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:07.053327 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-grwlh/perf-node-gather-daemonset-kr7sn" Jul 12 22:42:08.858050 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:08.857959 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h8cc7_3936ebea-d272-4f7c-bb93-f2386ec51915/manager/0.log" Jul 12 22:42:09.107475 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.107444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7bgfk_f65507a5-3e59-4ee5-bb70-23248e52acb7/manager/0.log" Jul 12 22:42:09.285806 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.285774 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6df67c4d8d-hk9sv_da1ca873-df0e-4f32-a757-dc2784b68972/manager/0.log" Jul 12 22:42:09.305431 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.305404 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vj2hh_b644c4a1-e77c-47f3-9ea4-2ec1ea55e32d/server/0.log" Jul 12 22:42:09.409721 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.409692 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2dhns_5ee07fe0-ed0d-4209-a0ad-7aa22cc346e2/manager/0.log" Jul 12 22:42:09.578553 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.578468 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d867dd548-rsdbp_bbf0987a-c7be-4b37-9aa7-cb729d6886ba/manager/0.log" Jul 12 22:42:09.599395 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:09.599362 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d867dd548-rsdbp_bbf0987a-c7be-4b37-9aa7-cb729d6886ba/copy-manifests/0.log" Jul 12 22:42:10.667916 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.667887 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c75d_ae0e1df6-a811-41c6-8856-ae0feeeef971/istio-proxy/0.log" Jul 12 22:42:10.800051 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.800020 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b84d88d8-s6s7s_dbaf50e2-97f6-49a9-9273-a35f8f3d8c44/kube-auth-proxy/0.log" Jul 12 22:42:10.833395 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.833368 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/0.log" Jul 12 22:42:10.892901 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.892866 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/1.log" Jul 12 22:42:10.969499 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.969411 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-262s7_6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f/payload-pre-processing/0.log" Jul 12 22:42:10.997747 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:10.997716 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-8nz58_411c6caf-3243-4906-bdd8-848fa8ce1b66/payload-processing/0.log" Jul 12 22:42:12.258303 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.258269 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44c75d_ae0e1df6-a811-41c6-8856-ae0feeeef971/istio-proxy/0.log" Jul 12 22:42:12.378317 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.378283 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66b84d88d8-s6s7s_dbaf50e2-97f6-49a9-9273-a35f8f3d8c44/kube-auth-proxy/0.log" Jul 12 22:42:12.412461 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.412411 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/0.log" Jul 12 22:42:12.475251 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.475201 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-twfkl_9e6b489e-069f-400d-b679-655002457efa/istio-proxy/1.log" Jul 12 22:42:12.503712 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.503676 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-262s7_6243477e-5ad1-4f2b-8a84-d1dfbcc4bf5f/payload-pre-processing/0.log" Jul 12 22:42:12.533016 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:12.532927 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-8nz58_411c6caf-3243-4906-bdd8-848fa8ce1b66/payload-processing/0.log" Jul 12 22:42:15.638425 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:15.638396 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-694pt_565820fb-1c32-4a23-a236-12adac158923/migrator/0.log" Jul 12 22:42:15.669917 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:15.669894 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-694pt_565820fb-1c32-4a23-a236-12adac158923/graceful-termination/0.log" Jul 12 22:42:17.047824 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.047790 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7bnnm_2f9b93ec-4582-4341-be8a-e777ff69340c/kube-multus/0.log" Jul 12 22:42:17.074465 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.074440 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/kube-multus-additional-cni-plugins/0.log" Jul 12 22:42:17.106905 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.106872 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/egress-router-binary-copy/0.log" Jul 12 22:42:17.133191 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.133162 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/cni-plugins/0.log" Jul 12 22:42:17.155989 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.155964 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/bond-cni-plugin/0.log" Jul 12 22:42:17.179332 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.179302 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/routeoverride-cni/0.log" Jul 12 22:42:17.202677 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.202650 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/whereabouts-cni-bincopy/0.log" Jul 12 22:42:17.222024 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.221996 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bzrhw_7d4a260a-963c-4a9d-938b-29ec4d903536/whereabouts-cni/0.log" Jul 12 22:42:17.600035 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.600004 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mtcjn_4841a61b-e23a-4fab-ab80-6705746aaf9a/network-metrics-daemon/0.log" Jul 12 22:42:17.616397 ip-10-0-132-6 kubenswrapper[2575]: I0712 22:42:17.616363 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mtcjn_4841a61b-e23a-4fab-ab80-6705746aaf9a/kube-rbac-proxy/0.log"