Jul 24 17:43:03.690680 ip-10-0-132-71 systemd[1]: Starting Kubernetes Kubelet... Jul 24 17:43:04.050318 ip-10-0-132-71 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:43:04.050318 ip-10-0-132-71 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 17:43:04.050318 ip-10-0-132-71 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:43:04.050318 ip-10-0-132-71 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 17:43:04.050318 ip-10-0-132-71 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 17:43:04.051931 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.051840 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 17:43:04.056640 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056623 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.056640 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056640 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056643 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056647 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056650 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056653 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056656 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056660 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056663 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056665 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056668 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056673 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056678 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056681 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056684 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056687 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056690 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056693 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056696 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056699 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.056710 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056702 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056705 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056708 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056711 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056714 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056716 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056720 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056723 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056726 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056728 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056731 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056733 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056736 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056738 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056741 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056743 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056746 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056749 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056751 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056754 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.057169 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056757 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056759 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056761 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056764 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056766 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056769 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056771 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056774 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056777 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056780 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056783 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056785 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056788 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056790 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056794 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056797 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056801 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056804 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056807 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056810 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.057700 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056812 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056815 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056818 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056821 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056824 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056826 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056829 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056832 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056834 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056837 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056839 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056842 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056844 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056847 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056850 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056853 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056855 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056858 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056860 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056862 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.058208 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056865 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056868 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056870 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056873 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056875 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056878 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056881 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056884 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056888 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056890 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056893 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056896 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056898 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056901 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056904 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056908 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056912 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056914 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.056917 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.058694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057624 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057632 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057635 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057637 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057640 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057643 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057646 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057648 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057652 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057655 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057657 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057661 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057664 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057666 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057669 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057672 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057675 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057677 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057680 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057683 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.059150 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057685 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057688 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057691 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057694 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057696 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057699 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057702 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057705 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057707 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057710 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057712 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057715 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057718 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057720 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057723 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057725 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057728 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057730 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057733 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057735 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.059672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057738 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057740 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057748 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057752 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057754 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057757 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057760 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057762 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057765 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057767 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057770 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057772 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057775 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057778 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057781 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057783 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057786 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057788 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057791 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.060193 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057794 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057796 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057799 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057802 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057806 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057810 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057813 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057816 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057819 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057821 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057824 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057827 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057829 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057832 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057834 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057838 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057841 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057845 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057849 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.060680 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057854 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057857 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057861 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057864 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057867 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057870 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057873 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057876 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057879 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057881 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057884 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057886 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057889 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057892 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057894 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057897 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057899 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057902 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057904 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057907 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.061149 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.057910 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058324 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058329 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058333 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058336 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058338 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058341 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058344 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058347 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058350 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058353 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058371 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058374 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058377 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058380 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058382 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058385 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058388 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058391 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058393 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.061650 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058396 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058398 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058401 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058403 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058406 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058409 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058411 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058414 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058417 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058420 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058424 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058428 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058431 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058434 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058436 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058439 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058441 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058443 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058446 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.062126 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058449 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058451 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058453 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058456 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058459 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058462 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058465 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058468 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058470 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058472 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058475 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058477 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058480 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058482 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058485 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058488 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058490 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058493 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058495 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058498 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.062607 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058500 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058503 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058505 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058508 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058511 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058513 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058516 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058518 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058521 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058523 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058526 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058529 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058531 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058534 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058536 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058538 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058541 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058544 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058546 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.063097 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058551 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058554 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058556 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058559 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058562 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058564 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058566 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058569 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058571 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058574 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058577 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058579 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058582 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058585 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058587 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058589 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058596 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058599 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058601 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058604 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.063579 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058608 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.058611 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058676 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058683 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058689 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058694 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058698 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058702 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058706 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058710 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058714 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058717 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058721 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058724 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058727 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058730 2566 flags.go:64] FLAG: --cgroup-root="" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058733 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058736 2566 flags.go:64] FLAG: --client-ca-file="" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058739 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058742 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058746 2566 flags.go:64] FLAG: --cluster-domain="" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058749 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058753 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058756 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 17:43:04.064056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058759 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058763 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058766 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058769 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058774 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058777 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058780 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058783 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058786 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058789 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058793 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058796 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058798 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058801 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058804 2566 flags.go:64] FLAG: --enable-server="true" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058807 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058812 2566 flags.go:64] FLAG: --event-burst="100" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058815 2566 flags.go:64] FLAG: --event-qps="50" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058818 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058822 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058824 2566 flags.go:64] FLAG: --eviction-hard="" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058830 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058833 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058836 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058839 2566 flags.go:64] FLAG: --eviction-soft="" Jul 24 17:43:04.064671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058842 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058845 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058848 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058851 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058854 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058857 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058860 2566 flags.go:64] FLAG: --feature-gates="" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058864 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058866 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058869 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058872 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058877 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058880 2566 flags.go:64] FLAG: --help="false" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058883 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058886 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058889 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058891 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058895 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058898 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058901 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058903 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058906 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058909 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 17:43:04.065272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058912 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058915 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058918 2566 flags.go:64] FLAG: --kube-reserved="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058921 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058924 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058928 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058931 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058934 2566 flags.go:64] FLAG: --lock-file="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058937 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058939 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058942 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058948 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058950 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058953 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058956 2566 flags.go:64] FLAG: --logging-format="text" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058959 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058963 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058965 2566 flags.go:64] FLAG: --manifest-url="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058968 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058973 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058977 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058981 2566 flags.go:64] FLAG: --max-pods="110" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058984 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058987 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058990 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 17:43:04.065915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058993 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058996 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.058999 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059002 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059008 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059011 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059014 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059017 2566 flags.go:64] FLAG: --pod-cidr="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059020 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059025 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059028 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059031 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059036 2566 flags.go:64] FLAG: --port="10250" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059039 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059042 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07f77fc8126275f38" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059045 2566 flags.go:64] FLAG: --qos-reserved="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059048 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059051 2566 flags.go:64] FLAG: --register-node="true" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059054 2566 flags.go:64] FLAG: --register-with-taints="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059057 2566 flags.go:64] FLAG: --registry-burst="10" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059060 2566 flags.go:64] FLAG: --registry-qps="5" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059063 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059066 2566 flags.go:64] FLAG: --reserved-memory="" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059070 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059073 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 17:43:04.066534 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059075 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059079 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059081 2566 flags.go:64] FLAG: --runonce="false" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059085 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059089 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059091 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059094 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059097 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059100 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059104 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059107 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059109 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059112 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059115 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059117 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059120 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059123 2566 flags.go:64] FLAG: --system-cgroups="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059126 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059131 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059136 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059139 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059143 2566 flags.go:64] FLAG: --tls-min-version="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059146 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059148 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059151 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 17:43:04.067158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059154 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059157 2566 flags.go:64] FLAG: --v="2" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059161 2566 flags.go:64] FLAG: --version="false" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059165 2566 flags.go:64] FLAG: --vmodule="" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059169 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.059172 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059294 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059300 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059304 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059308 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059311 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059316 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059319 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059322 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059325 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059327 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059331 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059335 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059338 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059340 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.067787 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059343 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059346 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059349 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059352 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059354 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059369 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059373 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059376 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059379 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059382 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059384 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059387 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059389 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059392 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059395 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059397 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059400 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059402 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059404 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059407 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.068269 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059410 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059412 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059415 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059418 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059421 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059423 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059433 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059437 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059440 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059443 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059445 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059448 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059450 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059453 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059456 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059458 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059461 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059464 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059468 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059471 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.068850 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059474 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059477 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059479 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059482 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059484 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059487 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059489 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059492 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059494 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059497 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059499 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059502 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059504 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059507 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059510 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059512 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059515 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059518 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059520 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059523 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.069382 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059525 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059528 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059530 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059533 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059535 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059538 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059541 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059543 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059546 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059548 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059552 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059554 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059557 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059559 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059562 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059565 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059567 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059570 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059573 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059575 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.069862 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059578 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059580 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059583 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059585 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.059587 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.060069 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.067967 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.067984 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.068613 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.068621 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068696 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068702 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068705 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068708 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.070346 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068711 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068716 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068720 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068723 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068726 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068729 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068732 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068734 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068737 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068740 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068743 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068746 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068749 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068751 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068754 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068756 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068759 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068762 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068764 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.070707 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068767 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068769 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068772 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068775 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068777 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068780 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068788 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068791 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068794 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068796 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068799 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068801 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068803 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068806 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068809 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068811 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068814 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068816 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068818 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.071212 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068821 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068823 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068826 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068828 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068830 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068833 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068835 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068838 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068840 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068843 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068847 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068850 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068853 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068857 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068860 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068863 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068865 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068868 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068871 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.071694 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068874 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068882 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068885 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068887 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068890 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068892 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068895 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068897 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068899 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068902 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068904 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068907 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068910 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068912 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068915 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068917 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068920 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068923 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068925 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.072156 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068928 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068936 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068938 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068941 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068943 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068946 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068950 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068953 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068956 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068959 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068961 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068963 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068967 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068970 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068972 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068981 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068984 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068987 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.072690 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.068989 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.068994 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069149 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069155 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069158 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069161 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069163 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069166 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069169 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069173 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069175 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069178 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069180 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069183 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 17:43:04.073289 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069185 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069188 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069190 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069193 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069195 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069198 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069200 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069203 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069205 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069207 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069210 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069212 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069216 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069220 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069223 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069225 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069234 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069236 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069239 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 17:43:04.073792 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069242 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069244 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069247 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069250 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069252 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069255 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069257 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069260 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069262 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069264 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069267 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069269 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069272 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069274 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069277 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069279 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069282 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069284 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069287 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069289 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 17:43:04.074325 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069291 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069294 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069296 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069300 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069303 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069307 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069309 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069312 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069315 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069317 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069326 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069329 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069346 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069350 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069353 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069372 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069377 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069380 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069383 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069386 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 17:43:04.074840 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069388 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069391 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069394 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069396 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069399 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069401 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069404 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069407 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069409 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069411 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069414 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069416 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069419 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069421 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069423 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069426 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069429 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069431 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069434 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069436 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 24 17:43:04.075312 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069439 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069441 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069443 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069452 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069455 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069457 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069460 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:04.069462 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.069467 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.069614 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.071735 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.074162 2566 server.go:1019] "Starting client certificate rotation" Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.074393 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 17:43:04.075836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.074439 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 17:43:04.076640 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.076622 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 17:43:04.077824 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.077805 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 17:43:04.092053 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.092026 2566 log.go:25] "Validated CRI v1 runtime API" Jul 24 17:43:04.094001 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.093988 2566 log.go:25] "Validated CRI v1 image API" Jul 24 17:43:04.094797 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.094782 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 17:43:04.096768 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.096749 2566 fs.go:135] Filesystem UUIDs: map[2b9b039a-1875-4f63-ab7c-ce4d2e0d2d38:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 be58cba5-556e-4b52-9649-4475c43fb51d:/dev/nvme0n1p4] Jul 24 17:43:04.096817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.096768 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 17:43:04.102536 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.102519 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 17:43:04.102623 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.102481 2566 manager.go:216] Machine: {Timestamp:2026-07-24 17:43:04.101369092 +0000 UTC m=+0.302601473 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110695 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218028e0a1d659fed23a1c62f886aa SystemUUID:ec218028-e0a1-d659-fed2-3a1c62f886aa BootID:9b7e25e3-735e-46c2-b100-4c66dac37fdb Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e0:d3:5b:2e:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e0:d3:5b:2e:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:e9:b9:bd:7e:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 17:43:04.102623 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.102626 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 17:43:04.102741 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.102728 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 17:43:04.106260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.106235 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 17:43:04.106431 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.106263 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-71.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 17:43:04.107314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.107304 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 17:43:04.107353 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.107318 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 17:43:04.107353 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.107326 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 17:43:04.107353 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.107340 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 17:43:04.108001 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.107991 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 17:43:04.109656 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109645 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 24 17:43:04.109776 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109767 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 17:43:04.109812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109804 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 24 17:43:04.109845 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109828 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 17:43:04.109845 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109841 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 17:43:04.109898 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109852 2566 kubelet.go:398] "Adding apiserver pod source" Jul 24 17:43:04.109898 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.109867 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 17:43:04.110308 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.110293 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 17:43:04.110425 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.110313 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 17:43:04.112387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.112346 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 17:43:04.115547 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.115522 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 17:43:04.115635 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.115571 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 17:43:04.117148 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117122 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 17:43:04.117148 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117143 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 17:43:04.117148 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117150 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117155 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117165 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117170 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117180 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117185 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117192 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117198 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117206 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 17:43:04.117282 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117212 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 17:43:04.117955 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117943 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 17:43:04.117993 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.117957 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 17:43:04.120090 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.120068 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvbcd" Jul 24 17:43:04.120205 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.120185 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-71.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 17:43:04.120679 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.120659 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-71.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 17:43:04.120733 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.120663 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 17:43:04.121949 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.121936 2566 server.go:1268] "Started kubelet" Jul 24 17:43:04.122040 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.122007 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 17:43:04.122090 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.122016 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 17:43:04.122144 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.122101 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 17:43:04.122880 ip-10-0-132-71 systemd[1]: Started Kubernetes Kubelet. Jul 24 17:43:04.123347 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.123331 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 17:43:04.124151 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.124136 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 24 17:43:04.127274 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.127253 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pvbcd" Jul 24 17:43:04.130022 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.130003 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 17:43:04.133332 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.132568 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-71.ec2.internal.18c54ab89edbc2c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-71.ec2.internal,UID:ip-10-0-132-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-71.ec2.internal,},FirstTimestamp:2026-07-24 17:43:04.121909953 +0000 UTC m=+0.323142333,LastTimestamp:2026-07-24 17:43:04.121909953 +0000 UTC m=+0.323142333,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-71.ec2.internal,}" Jul 24 17:43:04.134286 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.134261 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 17:43:04.134286 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.134272 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 17:43:04.134448 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.134397 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.134925 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.134908 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 17:43:04.134925 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.134918 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 17:43:04.135056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.134934 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 17:43:04.135056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135049 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 24 17:43:04.135056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135057 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 24 17:43:04.135424 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135406 2566 factory.go:153] Registering CRI-O factory Jul 24 17:43:04.135510 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135434 2566 factory.go:223] Registration of the crio container factory successfully Jul 24 17:43:04.135510 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135496 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 17:43:04.135510 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135505 2566 factory.go:55] Registering systemd factory Jul 24 17:43:04.135662 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135514 2566 factory.go:223] Registration of the systemd container factory successfully Jul 24 17:43:04.135662 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135542 2566 factory.go:103] Registering Raw factory Jul 24 17:43:04.135662 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.135556 2566 manager.go:1196] Started watching for new ooms in manager Jul 24 17:43:04.136049 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.136033 2566 manager.go:349] Starting recovery of all containers Jul 24 17:43:04.143423 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.143397 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:04.146340 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.146310 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-71.ec2.internal\" not found" node="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.148556 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.148534 2566 manager.go:354] Recovery completed Jul 24 17:43:04.150870 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.150849 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 24 17:43:04.154335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.154321 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.157123 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157107 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.157183 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157138 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.157183 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157149 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.157712 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157699 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 17:43:04.157712 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157706 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.157712 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157711 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 17:43:04.157853 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.157728 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 24 17:43:04.158509 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.158493 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.158565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.158525 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.158565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.158538 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.159324 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.159312 2566 policy_none.go:49] "None policy: Start" Jul 24 17:43:04.159444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.159328 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 17:43:04.159444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.159339 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 17:43:04.160445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.160431 2566 policy_none.go:47] "Start" Jul 24 17:43:04.203166 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203143 2566 manager.go:337] "Starting Device Plugin manager" Jul 24 17:43:04.203767 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.203190 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 17:43:04.203767 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203203 2566 server.go:87] "Starting device plugin registration server" Jul 24 17:43:04.203767 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203559 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 17:43:04.203767 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203575 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 17:43:04.204017 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203787 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 17:43:04.204017 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203864 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 17:43:04.204017 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.203873 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 17:43:04.204383 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.204333 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 17:43:04.204460 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.204417 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.241290 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.241251 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 17:43:04.242485 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.242461 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 17:43:04.242615 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.242504 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 17:43:04.242615 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.242528 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 17:43:04.242615 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.242563 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 17:43:04.244936 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.244918 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:04.304111 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.304025 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.305029 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.305008 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.305089 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.305038 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.305089 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.305049 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.305089 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.305071 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.311006 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.310985 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.311006 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.311008 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-71.ec2.internal\": node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.325851 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.325676 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.343563 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.343531 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal"] Jul 24 17:43:04.343674 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.343610 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.344584 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.344566 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.344665 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.344602 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.344665 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.344618 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.345833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.345821 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.345937 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.345922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.346001 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.345962 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.346603 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346588 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.346687 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346616 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.346687 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346631 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.346687 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346662 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.346832 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346694 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.346832 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.346709 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.347752 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.347739 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.347826 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.347763 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 17:43:04.348434 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.348419 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientMemory" Jul 24 17:43:04.348570 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.348443 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 17:43:04.348570 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.348453 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeHasSufficientPID" Jul 24 17:43:04.363973 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.363953 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-71.ec2.internal\" not found" node="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.368427 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.368411 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-71.ec2.internal\" not found" node="ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.426503 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.426478 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.436730 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.436709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.436788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.436742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.436788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.436760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96d0a8774c9d2bcc0171d5a7c66343bb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-71.ec2.internal\" (UID: \"96d0a8774c9d2bcc0171d5a7c66343bb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.526987 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.526950 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.537269 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96d0a8774c9d2bcc0171d5a7c66343bb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-71.ec2.internal\" (UID: \"96d0a8774c9d2bcc0171d5a7c66343bb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.537316 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.537316 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.537401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537343 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.537401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537351 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676e8d6ae334add5968a8c4c8fe78e30-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal\" (UID: \"676e8d6ae334add5968a8c4c8fe78e30\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.537465 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.537411 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96d0a8774c9d2bcc0171d5a7c66343bb-config\") pod \"kube-apiserver-proxy-ip-10-0-132-71.ec2.internal\" (UID: \"96d0a8774c9d2bcc0171d5a7c66343bb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.627767 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.627708 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.667274 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.667244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.671799 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.671780 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:04.728559 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.728523 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.829100 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.829071 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.929636 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:04.929566 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-71.ec2.internal\" not found" Jul 24 17:43:04.993654 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:04.993626 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:05.035561 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.035525 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" Jul 24 17:43:05.047917 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.047895 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 17:43:05.049551 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.049535 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" Jul 24 17:43:05.058975 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.058950 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 17:43:05.074241 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.074220 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 17:43:05.074412 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.074388 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 17:43:05.074470 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.074409 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 17:43:05.074470 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.074414 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 17:43:05.110721 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.110694 2566 apiserver.go:52] "Watching apiserver" Jul 24 17:43:05.127394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.127369 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 17:43:05.127803 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.127781 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qdzsl","kube-system/konnectivity-agent-kddtr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5","openshift-dns/node-resolver-t9nvd","openshift-image-registry/node-ca-zhwnj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal","openshift-multus/multus-wd2tj","openshift-multus/network-metrics-daemon-b6khh","kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal","openshift-cluster-node-tuning-operator/tuned-825rw","openshift-multus/multus-additional-cni-plugins-bq4cg","openshift-network-diagnostics/network-check-target-zrnmv","openshift-network-operator/iptables-alerter-x5bf8","openshift-ovn-kubernetes/ovnkube-node-4n45h"] Jul 24 17:43:05.129214 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.129196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:05.129311 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.129280 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:05.130612 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.130595 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.131764 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.131744 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.133004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.132952 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:38:04 +0000 UTC" deadline="2028-02-28 00:29:23.292724078 +0000 UTC" Jul 24 17:43:05.133111 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.133003 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13998h46m18.159725011s" Jul 24 17:43:05.134121 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.133671 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4bdwg\"" Jul 24 17:43:05.134121 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.133854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.134121 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.133955 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 17:43:05.134121 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.134088 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 17:43:05.135938 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.134529 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 17:43:05.135938 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.134761 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 17:43:05.135938 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.135106 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.135938 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.135111 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b4kjv\"" Jul 24 17:43:05.136837 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.136759 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 17:43:05.136837 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.136799 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.136982 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.136966 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 17:43:05.137065 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 17:43:05.137117 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137046 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.137168 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137148 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.137345 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137329 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.137449 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137365 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v4s4s\"" Jul 24 17:43:05.137596 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137581 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 17:43:05.137918 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137901 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.137975 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.137960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.138832 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.138815 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.139245 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.139229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.139544 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.139522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8xnsg\"" Jul 24 17:43:05.140455 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140384 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.140702 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt55f\" (UniqueName: \"kubernetes.io/projected/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-kube-api-access-qt55f\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.140823 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.140823 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140749 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 17:43:05.140823 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140691 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.140823 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.140795 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140723 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-slash\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140944 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-netns\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.140973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-bin\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11364763-f913-455b-9687-aa0a8bc528bc-tmp-dir\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq7dt\" (UniqueName: \"kubernetes.io/projected/11364763-f913-455b-9687-aa0a8bc528bc-kube-api-access-wq7dt\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.141099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxkn\" (UniqueName: \"kubernetes.io/projected/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-kube-api-access-4nxkn\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-host-slash\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-systemd-units\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-netd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141177 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-script-lib\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-host\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2254c944-7260-42a3-ba48-3c2dfea4695a-konnectivity-ca\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x99w4\" (UniqueName: \"kubernetes.io/projected/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-kube-api-access-x99w4\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-serviceca\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-iptables-alerter-script\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.141415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-config\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-systemd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11364763-f913-455b-9687-aa0a8bc528bc-hosts-file\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-kubelet\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-etc-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-ovn\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-node-log\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-log-socket\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovn-node-metrics-cert\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2254c944-7260-42a3-ba48-3c2dfea4695a-agent-certs\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-var-lib-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.141805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-env-overrides\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.142196 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.141851 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 17:43:05.142196 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.142062 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n5tzf\"" Jul 24 17:43:05.142313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.142298 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 17:43:05.142624 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.142601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.143185 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.143165 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.143284 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.143233 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.143350 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.143299 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5mznr\"" Jul 24 17:43:05.144169 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.144153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.145146 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.145129 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.145561 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.145520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.145654 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.145609 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:05.145713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.145660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.145759 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.145725 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sdkc7\"" Jul 24 17:43:05.146575 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.146557 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 17:43:05.146715 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.146700 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 17:43:05.147096 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.146783 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 17:43:05.147096 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.146811 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxh8b\"" Jul 24 17:43:05.147096 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.146949 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.149770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.149586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 17:43:05.149770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.149635 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 17:43:05.149770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.149648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jcfc6\"" Jul 24 17:43:05.149770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.149663 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 17:43:05.168903 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.168881 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9zwl" Jul 24 17:43:05.177899 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.177871 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9zwl" Jul 24 17:43:05.236112 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.236090 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 17:43:05.242130 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-host\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.242219 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242138 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2254c944-7260-42a3-ba48-3c2dfea4695a-konnectivity-ca\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.242219 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.242219 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lvrc\" (UniqueName: \"kubernetes.io/projected/d941add4-2f21-41d2-8d48-2fad9b133c31-kube-api-access-8lvrc\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-host\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242223 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-registration-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-host\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-multus\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.242333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-systemd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242359 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-systemd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242432 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpsp\" (UniqueName: \"kubernetes.io/projected/6cc1502b-f19f-416f-94ef-04e577a87b67-kube-api-access-9kpsp\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-etc-tuned\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-os-release\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242518 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11364763-f913-455b-9687-aa0a8bc528bc-hosts-file\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.242560 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-kubelet\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242573 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-kubernetes\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11364763-f913-455b-9687-aa0a8bc528bc-hosts-file\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242616 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-kubelet\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-device-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242674 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-ovn\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovn-node-metrics-cert\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242719 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cnibin\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqprw\" (UniqueName: \"kubernetes.io/projected/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kube-api-access-pqprw\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-ovn\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.242793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2254c944-7260-42a3-ba48-3c2dfea4695a-konnectivity-ca\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2254c944-7260-42a3-ba48-3c2dfea4695a-agent-certs\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242844 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-systemd\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242872 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jptsm\" (UniqueName: \"kubernetes.io/projected/9e1b49da-53a7-4474-995f-75585c064967-kube-api-access-jptsm\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-cni-binary-copy\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-slash\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-run\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-slash\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.242993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-var-lib-kubelet\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243033 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-cnibin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-bin\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243047 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243089 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-bin\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-netd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-script-lib\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-conf\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243215 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-cni-netd\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.243255 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243236 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-tmp\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-dbus\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243289 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-k8s-cni-cncf-io\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x99w4\" (UniqueName: \"kubernetes.io/projected/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-kube-api-access-x99w4\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-kubelet-config\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-etc-selinux\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-serviceca\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-iptables-alerter-script\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-config\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-sys\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243600 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-sys-fs\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-hostroot\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxw2r\" (UniqueName: \"kubernetes.io/projected/7bb784c5-92b8-4ad5-9041-981977aa7cf3-kube-api-access-hxw2r\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.244208 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243826 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-run-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-script-lib\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-socket-dir-parent\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-kubelet\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243954 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-system-cni-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.243983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-etc-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-node-log\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-log-socket\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244054 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovnkube-config\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-modprobe-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysconfig\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-lib-modules\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244124 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-iptables-alerter-script\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-log-socket\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-system-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244177 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-node-log\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244181 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-etc-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-multus-certs\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-var-lib-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-env-overrides\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244313 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-var-lib-openvswitch\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-conf-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-etc-kubernetes\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244562 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-socket-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt55f\" (UniqueName: \"kubernetes.io/projected/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-kube-api-access-qt55f\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-netns\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-bin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-env-overrides\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244731 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11364763-f913-455b-9687-aa0a8bc528bc-tmp-dir\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.245812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244767 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq7dt\" (UniqueName: \"kubernetes.io/projected/11364763-f913-455b-9687-aa0a8bc528bc-kube-api-access-wq7dt\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxkn\" (UniqueName: \"kubernetes.io/projected/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-kube-api-access-4nxkn\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244812 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-host-run-netns\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-host-slash\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-systemd-units\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-os-release\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-netns\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-daemon-config\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.244985 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-host-slash\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.245071 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-systemd-units\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.245081 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11364763-f913-455b-9687-aa0a8bc528bc-tmp-dir\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.246713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.245409 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-serviceca\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.247296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.247087 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-ovn-node-metrics-cert\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.247296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.247156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2254c944-7260-42a3-ba48-3c2dfea4695a-agent-certs\") pod \"konnectivity-agent-kddtr\" (UID: \"2254c944-7260-42a3-ba48-3c2dfea4695a\") " pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.248641 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.248609 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676e8d6ae334add5968a8c4c8fe78e30.slice/crio-234a102b2e6993ba6ecfb559ecb1e662d27144c85d0d8d8a6287a06e5c86e91f WatchSource:0}: Error finding container 234a102b2e6993ba6ecfb559ecb1e662d27144c85d0d8d8a6287a06e5c86e91f: Status 404 returned error can't find the container with id 234a102b2e6993ba6ecfb559ecb1e662d27144c85d0d8d8a6287a06e5c86e91f Jul 24 17:43:05.248915 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.248877 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d0a8774c9d2bcc0171d5a7c66343bb.slice/crio-1c9a971388c3d666d9b75aeeb36307200a1ad146f2b8d79cf4f88d6e76c1d8a9 WatchSource:0}: Error finding container 1c9a971388c3d666d9b75aeeb36307200a1ad146f2b8d79cf4f88d6e76c1d8a9: Status 404 returned error can't find the container with id 1c9a971388c3d666d9b75aeeb36307200a1ad146f2b8d79cf4f88d6e76c1d8a9 Jul 24 17:43:05.250521 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.250500 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:05.250601 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.250529 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:05.250601 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.250544 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:05.250691 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.250610 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:05.750589757 +0000 UTC m=+1.951822131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:05.252477 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.252461 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x99w4\" (UniqueName: \"kubernetes.io/projected/ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd-kube-api-access-x99w4\") pod \"ovnkube-node-4n45h\" (UID: \"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.253397 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.253371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt55f\" (UniqueName: \"kubernetes.io/projected/c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef-kube-api-access-qt55f\") pod \"iptables-alerter-x5bf8\" (UID: \"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef\") " pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.254248 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.254225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxkn\" (UniqueName: \"kubernetes.io/projected/3ea17a03-d149-4a7c-9f03-47f288bf3f3c-kube-api-access-4nxkn\") pod \"node-ca-zhwnj\" (UID: \"3ea17a03-d149-4a7c-9f03-47f288bf3f3c\") " pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.254707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.254662 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq7dt\" (UniqueName: \"kubernetes.io/projected/11364763-f913-455b-9687-aa0a8bc528bc-kube-api-access-wq7dt\") pod \"node-resolver-t9nvd\" (UID: \"11364763-f913-455b-9687-aa0a8bc528bc\") " pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.255294 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.255277 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 17:43:05.345551 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-multus-certs\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-conf-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-etc-kubernetes\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-socket-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-bin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-multus-certs\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-os-release\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-etc-kubernetes\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345704 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-bin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-conf-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-netns\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-daemon-config\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.345763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345768 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-os-release\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lvrc\" (UniqueName: \"kubernetes.io/projected/d941add4-2f21-41d2-8d48-2fad9b133c31-kube-api-access-8lvrc\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-netns\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-registration-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-host\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345892 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-multus\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345903 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-registration-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.345934 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpsp\" (UniqueName: \"kubernetes.io/projected/6cc1502b-f19f-416f-94ef-04e577a87b67-kube-api-access-9kpsp\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345954 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-host\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345962 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-cni-multus\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.345969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-etc-tuned\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.346018 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:05.845992817 +0000 UTC m=+2.047225203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.346036 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:05.346475 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.346082 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:05.846066015 +0000 UTC m=+2.047298383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346154 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-os-release\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346202 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-kubernetes\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-os-release\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-kubernetes\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346314 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-device-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346375 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-daemon-config\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cnibin\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346419 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-device-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346454 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cnibin\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqprw\" (UniqueName: \"kubernetes.io/projected/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kube-api-access-pqprw\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-systemd\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.347363 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346543 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jptsm\" (UniqueName: \"kubernetes.io/projected/9e1b49da-53a7-4474-995f-75585c064967-kube-api-access-jptsm\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-cni-binary-copy\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346585 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-systemd\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346597 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346645 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-run\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-var-lib-kubelet\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346693 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-cnibin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-conf\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-tmp\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346787 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-var-lib-kubelet\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-dbus\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-k8s-cni-cncf-io\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346848 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-dbus\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346886 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-kubelet-config\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-binary-copy\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-run\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346791 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-cnibin\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.348309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-etc-selinux\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-sys\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysctl-conf\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6e14a3c-14e7-472b-9452-5b8f01744623-kubelet-config\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.346985 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347003 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-etc-selinux\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-sys-fs\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347029 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-sys\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347052 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-run-k8s-cni-cncf-io\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-hostroot\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-sys-fs\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347088 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d941add4-2f21-41d2-8d48-2fad9b133c31-cni-binary-copy\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347106 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-hostroot\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347112 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxw2r\" (UniqueName: \"kubernetes.io/projected/7bb784c5-92b8-4ad5-9041-981977aa7cf3-kube-api-access-hxw2r\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-socket-dir-parent\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-kubelet\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-system-cni-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347261 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-modprobe-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347268 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-multus-socket-dir-parent\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-host-var-lib-kubelet\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysconfig\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347369 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7bb784c5-92b8-4ad5-9041-981977aa7cf3-system-cni-dir\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-lib-modules\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347487 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-modprobe-d\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347492 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-system-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347513 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/433f7dcb-b079-4507-bb70-854fe8fdeaf4-socket-dir\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347549 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d941add4-2f21-41d2-8d48-2fad9b133c31-system-cni-dir\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347558 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-etc-sysconfig\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347557 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7bb784c5-92b8-4ad5-9041-981977aa7cf3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.347622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e1b49da-53a7-4474-995f-75585c064967-lib-modules\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.349156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-etc-tuned\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.349785 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.349255 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e1b49da-53a7-4474-995f-75585c064967-tmp\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.359221 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.359189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpsp\" (UniqueName: \"kubernetes.io/projected/6cc1502b-f19f-416f-94ef-04e577a87b67-kube-api-access-9kpsp\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.359353 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.359185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lvrc\" (UniqueName: \"kubernetes.io/projected/d941add4-2f21-41d2-8d48-2fad9b133c31-kube-api-access-8lvrc\") pod \"multus-wd2tj\" (UID: \"d941add4-2f21-41d2-8d48-2fad9b133c31\") " pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.359795 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.359775 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxw2r\" (UniqueName: \"kubernetes.io/projected/7bb784c5-92b8-4ad5-9041-981977aa7cf3-kube-api-access-hxw2r\") pod \"multus-additional-cni-plugins-bq4cg\" (UID: \"7bb784c5-92b8-4ad5-9041-981977aa7cf3\") " pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.359909 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.359893 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jptsm\" (UniqueName: \"kubernetes.io/projected/9e1b49da-53a7-4474-995f-75585c064967-kube-api-access-jptsm\") pod \"tuned-825rw\" (UID: \"9e1b49da-53a7-4474-995f-75585c064967\") " pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.360211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.360188 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqprw\" (UniqueName: \"kubernetes.io/projected/433f7dcb-b079-4507-bb70-854fe8fdeaf4-kube-api-access-pqprw\") pod \"aws-ebs-csi-driver-node-lpvm5\" (UID: \"433f7dcb-b079-4507-bb70-854fe8fdeaf4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.463742 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.463630 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:05.473106 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.473076 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2254c944_7260_42a3_ba48_3c2dfea4695a.slice/crio-350742a8ad3aea9c0ec319fe5945fe8654f3da037f8a2655cb92952722590160 WatchSource:0}: Error finding container 350742a8ad3aea9c0ec319fe5945fe8654f3da037f8a2655cb92952722590160: Status 404 returned error can't find the container with id 350742a8ad3aea9c0ec319fe5945fe8654f3da037f8a2655cb92952722590160 Jul 24 17:43:05.481702 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.481679 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:05.490130 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.490060 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1ae4f5_9036_4c2e_9cd7_97fa2da937fd.slice/crio-c54d61922ee704be1fa66eaadc04e6ab8c0739a043bff27304119d31f560e009 WatchSource:0}: Error finding container c54d61922ee704be1fa66eaadc04e6ab8c0739a043bff27304119d31f560e009: Status 404 returned error can't find the container with id c54d61922ee704be1fa66eaadc04e6ab8c0739a043bff27304119d31f560e009 Jul 24 17:43:05.494478 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.494455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t9nvd" Jul 24 17:43:05.499571 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.499553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zhwnj" Jul 24 17:43:05.502218 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.502182 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11364763_f913_455b_9687_aa0a8bc528bc.slice/crio-c4797abd9b4e7a9a868a09228fee68fca0354c242768265ca0ad5a566c906e01 WatchSource:0}: Error finding container c4797abd9b4e7a9a868a09228fee68fca0354c242768265ca0ad5a566c906e01: Status 404 returned error can't find the container with id c4797abd9b4e7a9a868a09228fee68fca0354c242768265ca0ad5a566c906e01 Jul 24 17:43:05.507719 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.507694 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea17a03_d149_4a7c_9f03_47f288bf3f3c.slice/crio-cdfccfd7cb95bb8bbf905ce9abc71b1c00f1119d1cceccb945cc590274d26ef4 WatchSource:0}: Error finding container cdfccfd7cb95bb8bbf905ce9abc71b1c00f1119d1cceccb945cc590274d26ef4: Status 404 returned error can't find the container with id cdfccfd7cb95bb8bbf905ce9abc71b1c00f1119d1cceccb945cc590274d26ef4 Jul 24 17:43:05.518269 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.518247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5bf8" Jul 24 17:43:05.528564 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.528539 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc277df9f_e88e_4aa8_8d7c_b4fa9d4779ef.slice/crio-32002b38e95011b120f5b5c17f75e9c8c314e56d9e9a21341ebe850c9ee35aca WatchSource:0}: Error finding container 32002b38e95011b120f5b5c17f75e9c8c314e56d9e9a21341ebe850c9ee35aca: Status 404 returned error can't find the container with id 32002b38e95011b120f5b5c17f75e9c8c314e56d9e9a21341ebe850c9ee35aca Jul 24 17:43:05.539033 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.539010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wd2tj" Jul 24 17:43:05.546932 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.546906 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd941add4_2f21_41d2_8d48_2fad9b133c31.slice/crio-7cd69f9492be37e17d5e74318a45935f29ba81a0a9b905604075524b46292378 WatchSource:0}: Error finding container 7cd69f9492be37e17d5e74318a45935f29ba81a0a9b905604075524b46292378: Status 404 returned error can't find the container with id 7cd69f9492be37e17d5e74318a45935f29ba81a0a9b905604075524b46292378 Jul 24 17:43:05.549205 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.549185 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-825rw" Jul 24 17:43:05.556478 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.556456 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" Jul 24 17:43:05.556613 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.556591 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1b49da_53a7_4474_995f_75585c064967.slice/crio-d1410b496acd83661fd335c6d7d408d9478d3e352c869c3d385e3554b399a58a WatchSource:0}: Error finding container d1410b496acd83661fd335c6d7d408d9478d3e352c869c3d385e3554b399a58a: Status 404 returned error can't find the container with id d1410b496acd83661fd335c6d7d408d9478d3e352c869c3d385e3554b399a58a Jul 24 17:43:05.561792 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.561772 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" Jul 24 17:43:05.570651 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:05.570631 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433f7dcb_b079_4507_bb70_854fe8fdeaf4.slice/crio-dbc85155775d88bbccf1c163ae8bff4b074c0cb6ee425b635dd205c604647855 WatchSource:0}: Error finding container dbc85155775d88bbccf1c163ae8bff4b074c0cb6ee425b635dd205c604647855: Status 404 returned error can't find the container with id dbc85155775d88bbccf1c163ae8bff4b074c0cb6ee425b635dd205c604647855 Jul 24 17:43:05.659129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.659092 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:05.750895 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.750799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:05.751051 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.751025 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:05.751106 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.751059 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:05.751106 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.751074 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:05.751195 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.751156 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:06.751135318 +0000 UTC m=+2.952367711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:05.851431 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.851394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:05.851631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.851507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:05.851703 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.851636 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:05.851703 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.851693 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:05.851808 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.851708 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:06.851687306 +0000 UTC m=+3.052919695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:05.851808 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:05.851745 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:06.851733218 +0000 UTC m=+3.052965603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:05.926052 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:05.926011 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:06.184647 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.184523 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:38:05 +0000 UTC" deadline="2028-04-26 14:01:15.392982672 +0000 UTC" Jul 24 17:43:06.184647 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.184593 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15404h18m9.208395318s" Jul 24 17:43:06.246833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.246735 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:06.247046 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.246875 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:06.262759 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.262692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"c54d61922ee704be1fa66eaadc04e6ab8c0739a043bff27304119d31f560e009"} Jul 24 17:43:06.279378 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.279285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" event={"ID":"96d0a8774c9d2bcc0171d5a7c66343bb","Type":"ContainerStarted","Data":"1c9a971388c3d666d9b75aeeb36307200a1ad146f2b8d79cf4f88d6e76c1d8a9"} Jul 24 17:43:06.312648 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.312553 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" event={"ID":"676e8d6ae334add5968a8c4c8fe78e30","Type":"ContainerStarted","Data":"234a102b2e6993ba6ecfb559ecb1e662d27144c85d0d8d8a6287a06e5c86e91f"} Jul 24 17:43:06.316565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.316527 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" event={"ID":"433f7dcb-b079-4507-bb70-854fe8fdeaf4","Type":"ContainerStarted","Data":"dbc85155775d88bbccf1c163ae8bff4b074c0cb6ee425b635dd205c604647855"} Jul 24 17:43:06.323316 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.323214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerStarted","Data":"1481614c5b3795b6f23c3487faf0d13806e9022281e73f37a0f5d2870f5da752"} Jul 24 17:43:06.347073 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.347027 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wd2tj" event={"ID":"d941add4-2f21-41d2-8d48-2fad9b133c31","Type":"ContainerStarted","Data":"7cd69f9492be37e17d5e74318a45935f29ba81a0a9b905604075524b46292378"} Jul 24 17:43:06.372598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.372514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kddtr" event={"ID":"2254c944-7260-42a3-ba48-3c2dfea4695a","Type":"ContainerStarted","Data":"350742a8ad3aea9c0ec319fe5945fe8654f3da037f8a2655cb92952722590160"} Jul 24 17:43:06.417466 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.416698 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-825rw" event={"ID":"9e1b49da-53a7-4474-995f-75585c064967","Type":"ContainerStarted","Data":"d1410b496acd83661fd335c6d7d408d9478d3e352c869c3d385e3554b399a58a"} Jul 24 17:43:06.424172 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.424130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5bf8" event={"ID":"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef","Type":"ContainerStarted","Data":"32002b38e95011b120f5b5c17f75e9c8c314e56d9e9a21341ebe850c9ee35aca"} Jul 24 17:43:06.436499 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.436383 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhwnj" event={"ID":"3ea17a03-d149-4a7c-9f03-47f288bf3f3c","Type":"ContainerStarted","Data":"cdfccfd7cb95bb8bbf905ce9abc71b1c00f1119d1cceccb945cc590274d26ef4"} Jul 24 17:43:06.454458 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.454383 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t9nvd" event={"ID":"11364763-f913-455b-9687-aa0a8bc528bc","Type":"ContainerStarted","Data":"c4797abd9b4e7a9a868a09228fee68fca0354c242768265ca0ad5a566c906e01"} Jul 24 17:43:06.632855 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.632654 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:06.721428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.721327 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 17:43:06.760643 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.760075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:06.760643 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.760269 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:06.760643 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.760294 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:06.760643 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.760308 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:06.760643 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.760374 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:08.760353095 +0000 UTC m=+4.961585469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:06.861232 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.861186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:06.861431 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:06.861245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:06.861431 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.861425 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:06.861582 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.861493 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:08.861472 +0000 UTC m=+5.062704370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:06.861975 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.861935 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:06.862081 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:06.861997 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:08.861980739 +0000 UTC m=+5.063213109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:07.185518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:07.185418 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:38:05 +0000 UTC" deadline="2028-04-12 17:52:51.162488764 +0000 UTC" Jul 24 17:43:07.185518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:07.185463 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15072h9m43.977030465s" Jul 24 17:43:07.243171 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:07.243137 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:07.243365 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:07.243283 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:07.243913 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:07.243734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:07.243913 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:07.243835 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:08.247515 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:08.247480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:08.248161 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.247626 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:08.778129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:08.778086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:08.778367 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.778344 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:08.778471 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.778382 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:08.778471 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.778400 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:08.778596 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.778495 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:12.778470545 +0000 UTC m=+8.979702930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:08.879571 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:08.879471 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:08.879571 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:08.879535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:08.879800 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.879677 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:08.879800 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.879748 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:12.879725749 +0000 UTC m=+9.080958123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:08.880486 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.880180 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:08.880486 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:08.880245 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:12.880225124 +0000 UTC m=+9.081457499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:09.242830 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:09.242744 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:09.243002 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:09.242906 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:09.243315 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:09.243294 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:09.243492 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:09.243469 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:10.246207 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:10.245643 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:10.246207 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:10.245782 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:11.243338 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:11.243296 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:11.243541 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:11.243453 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:11.243625 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:11.243571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:11.243730 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:11.243702 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:12.244416 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:12.244338 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:12.244926 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.244503 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:12.815478 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:12.815433 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:12.815691 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.815619 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:12.815691 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.815641 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:12.815691 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.815656 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:12.815880 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.815727 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:20.815705665 +0000 UTC m=+17.016938054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:12.916442 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:12.916398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:12.916616 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:12.916461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:12.916685 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.916644 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:12.916734 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.916715 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:20.916692111 +0000 UTC m=+17.117924499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:12.917182 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.917160 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:12.917290 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:12.917227 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:20.917210095 +0000 UTC m=+17.118442463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:13.242837 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:13.242751 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:13.242837 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:13.242803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:13.243059 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:13.242891 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:13.243059 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:13.243042 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:14.243983 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:14.243948 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:14.244468 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:14.244060 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:15.244517 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:15.243696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:15.244517 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:15.243849 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:15.244517 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:15.244230 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:15.244517 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:15.244345 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:16.245888 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:16.245860 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:16.246299 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:16.245967 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:17.243019 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:17.242968 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:17.243019 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:17.243006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:17.243275 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:17.243108 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:17.243275 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:17.243243 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:18.245863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:18.245829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:18.246316 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:18.245951 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:19.243059 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:19.243023 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:19.243205 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:19.243062 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:19.243205 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:19.243175 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:19.243353 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:19.243311 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:20.246245 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:20.246201 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:20.246653 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.246306 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:20.873641 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:20.873598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:20.873835 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.873804 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:20.873835 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.873829 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:20.873933 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.873841 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:20.873933 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.873907 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:36.873888171 +0000 UTC m=+33.075120554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:20.974178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:20.974133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:20.974348 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:20.974187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:20.974348 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.974284 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:20.974348 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.974333 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:20.974506 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.974355 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:36.974333373 +0000 UTC m=+33.175565762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:20.974506 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:20.974398 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:36.97438107 +0000 UTC m=+33.175613439 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:21.243613 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:21.243545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:21.243733 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:21.243545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:21.243733 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:21.243658 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:21.243733 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:21.243707 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:22.243837 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:22.243802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:22.244408 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:22.243948 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:23.243202 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:23.243160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:23.243413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:23.243167 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:23.243413 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:23.243272 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:23.243519 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:23.243413 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:24.244374 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:24.244323 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:24.244839 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:24.244465 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:25.242982 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:25.242942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:25.243149 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:25.243065 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:25.243149 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:25.243122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:25.243275 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:25.243251 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:26.243995 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.243823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:26.244347 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:26.244090 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:26.517779 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.517657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"b9173c78422cc925bbcd4cd94394597bd6dddc9fe6549a389b804c0880bbf9c4"} Jul 24 17:43:26.517868 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.517824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"56e6e63aaa5478f680d5f458abc836a61e500bbdaf575c0e5b29d2395da664c1"} Jul 24 17:43:26.519125 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.519101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" event={"ID":"96d0a8774c9d2bcc0171d5a7c66343bb","Type":"ContainerStarted","Data":"8e459c42d0658e7f4653441177458998f8c4a0a2e12a721639f8b6f868b3f3bc"} Jul 24 17:43:26.520793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.520765 2566 generic.go:356] "Generic (PLEG): container finished" podID="676e8d6ae334add5968a8c4c8fe78e30" containerID="c035367670f1c034797e8bbd81a36513be0fa2fe762b71bd089d8c35fa9d8b3c" exitCode=0 Jul 24 17:43:26.520917 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.520839 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" event={"ID":"676e8d6ae334add5968a8c4c8fe78e30","Type":"ContainerDied","Data":"c035367670f1c034797e8bbd81a36513be0fa2fe762b71bd089d8c35fa9d8b3c"} Jul 24 17:43:26.522415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.522392 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" event={"ID":"433f7dcb-b079-4507-bb70-854fe8fdeaf4","Type":"ContainerStarted","Data":"95f06e8940a9afa1fba6b5b527ac725e2fcb1098c8339c7ec610e1e64504842f"} Jul 24 17:43:26.524175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.524059 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="e4d179f1f67552e90a6d2eb4c14c1f0f390d282f33ba12f3af020d939d179e6e" exitCode=0 Jul 24 17:43:26.524175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.524137 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"e4d179f1f67552e90a6d2eb4c14c1f0f390d282f33ba12f3af020d939d179e6e"} Jul 24 17:43:26.525974 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.525942 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wd2tj" event={"ID":"d941add4-2f21-41d2-8d48-2fad9b133c31","Type":"ContainerStarted","Data":"cca39918a9ac3bd4660acde7630221f382aa2f9692e05fcd530ab9d4d0ed4b47"} Jul 24 17:43:26.528171 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.528142 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kddtr" event={"ID":"2254c944-7260-42a3-ba48-3c2dfea4695a","Type":"ContainerStarted","Data":"f27b02dd2cd9aa73c5ad52898b8bf3e9fe5303dc7966e286f9e2823fbdecc386"} Jul 24 17:43:26.529651 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.529632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-825rw" event={"ID":"9e1b49da-53a7-4474-995f-75585c064967","Type":"ContainerStarted","Data":"744eae95791a00def939f025499057bdc88f1e9cf6b8f31a5ea5dbad8e5c80d6"} Jul 24 17:43:26.531007 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.530982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zhwnj" event={"ID":"3ea17a03-d149-4a7c-9f03-47f288bf3f3c","Type":"ContainerStarted","Data":"82d39c9d1485991bc4293af70c1fb589196db3530bd0873ec0b9eaa9df0ec163"} Jul 24 17:43:26.532134 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.532117 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t9nvd" event={"ID":"11364763-f913-455b-9687-aa0a8bc528bc","Type":"ContainerStarted","Data":"70ed312093c2ad99b60616aff59d0bf699eca6beff68ff31b8d3c18a34df3129"} Jul 24 17:43:26.534821 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.534778 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-71.ec2.internal" podStartSLOduration=21.534764332 podStartE2EDuration="21.534764332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.534581402 +0000 UTC m=+22.735813792" watchObservedRunningTime="2026-07-24 17:43:26.534764332 +0000 UTC m=+22.735996726" Jul 24 17:43:26.568231 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.568191 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wd2tj" podStartSLOduration=1.583407274 podStartE2EDuration="21.568176037s" totalImagesPullingTime="19.984768763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.568033249 +0000 UTC m=+22.769265639" watchObservedRunningTime="2026-07-24 17:43:26.568176037 +0000 UTC m=+22.769408426" Jul 24 17:43:26.584852 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.584801 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-825rw" podStartSLOduration=1.6216558829999999 podStartE2EDuration="21.584781271s" totalImagesPullingTime="19.963125388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.584721732 +0000 UTC m=+22.785954120" watchObservedRunningTime="2026-07-24 17:43:26.584781271 +0000 UTC m=+22.786013676" Jul 24 17:43:26.624126 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.623936 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kddtr" podStartSLOduration=2.604089518 podStartE2EDuration="22.623921223s" totalImagesPullingTime="20.019831705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.623541275 +0000 UTC m=+22.824773664" watchObservedRunningTime="2026-07-24 17:43:26.623921223 +0000 UTC m=+22.825153612" Jul 24 17:43:26.643338 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.643296 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t9nvd" podStartSLOduration=2.628057008 podStartE2EDuration="22.643280144s" totalImagesPullingTime="20.015223136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.642915266 +0000 UTC m=+22.844147656" watchObservedRunningTime="2026-07-24 17:43:26.643280144 +0000 UTC m=+22.844512532" Jul 24 17:43:26.659210 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:26.659170 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zhwnj" podStartSLOduration=2.6740488559999998 podStartE2EDuration="22.659157467s" totalImagesPullingTime="19.985108611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:26.658669038 +0000 UTC m=+22.859901427" watchObservedRunningTime="2026-07-24 17:43:26.659157467 +0000 UTC m=+22.860389855" Jul 24 17:43:27.242968 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.242886 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:27.242968 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.242926 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:27.243164 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:27.243009 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:27.243164 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:27.243122 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:27.536754 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.536661 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5bf8" event={"ID":"c277df9f-e88e-4aa8-8d7c-b4fa9d4779ef","Type":"ContainerStarted","Data":"b3ec8349248226852ac8466794111b4a4055453ae3a7eee1e72ddc0ca7c1e573"} Jul 24 17:43:27.540584 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.540552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"71ca48b22f437f89969767ca78d741f8a29571b86225abd2f82ff8ea997a17ba"} Jul 24 17:43:27.540808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.540639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"d4ad67eb11f03d38958b9199af22c44d092c3dd46e0348de0bb5458c3dc74e83"} Jul 24 17:43:27.540808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.540688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"29d5f5baf5534cd20ea4afb72c701119a93442bff0b3dcadb7c1a02f01c6eaff"} Jul 24 17:43:27.540808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.540719 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"bb0712f2aae1637bb1d270ece6c82d55249e26c31fe85332e4603d3f04418f50"} Jul 24 17:43:27.542498 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.542416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" event={"ID":"676e8d6ae334add5968a8c4c8fe78e30","Type":"ContainerStarted","Data":"1a90ae60dd3f78062328515b60628485ac3d432e84c03b35bba5ba37452f5c72"} Jul 24 17:43:27.554038 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.553996 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x5bf8" podStartSLOduration=3.55851706 podStartE2EDuration="23.553979436s" totalImagesPullingTime="19.995462376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:27.553719158 +0000 UTC m=+23.754951548" watchObservedRunningTime="2026-07-24 17:43:27.553979436 +0000 UTC m=+23.755211843" Jul 24 17:43:27.570546 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.570493 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-71.ec2.internal" podStartSLOduration=22.570475709 podStartE2EDuration="22.570475709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:27.569903751 +0000 UTC m=+23.771136145" watchObservedRunningTime="2026-07-24 17:43:27.570475709 +0000 UTC m=+23.771708099" Jul 24 17:43:27.790031 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:27.789969 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 17:43:28.219200 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:28.219049 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:43:27.789990588Z","UUID":"c66dc86f-f1fc-47de-ae81-a0a9c323254b","Handler":null,"Name":"","Endpoint":""} Jul 24 17:43:28.220708 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:28.220687 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 17:43:28.220841 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:28.220741 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 17:43:28.243329 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:28.243304 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:28.243484 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:28.243437 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:28.547705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:28.547295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" event={"ID":"433f7dcb-b079-4507-bb70-854fe8fdeaf4","Type":"ContainerStarted","Data":"01b35ecb12d67be345899d9fa1f4fea1d7143640c4f72043f8739610bd7d554a"} Jul 24 17:43:29.243446 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.243407 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:29.243446 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.243443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:29.243631 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:29.243518 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:29.243667 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:29.243633 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:29.344976 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.344878 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:29.345661 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.345638 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:29.553775 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.553738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"8794d17a341364ae1eb53d75ae9eaedc53a628fa759bf2ad0c06f4c9e6024273"} Jul 24 17:43:29.554175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.553832 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:29.554454 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:29.554432 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kddtr" Jul 24 17:43:30.243099 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:30.243067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:30.243289 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:30.243212 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:31.243501 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.243478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:31.243890 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.243512 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:31.243890 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:31.243587 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:31.243890 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:31.243678 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:31.561238 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.561137 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" event={"ID":"ce1ae4f5-9036-4c2e-9cd7-97fa2da937fd","Type":"ContainerStarted","Data":"8aa499c3a327086095ab6d95db2cb6a4bb26ec08d33fce55badfe59599f1b95e"} Jul 24 17:43:31.561409 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.561391 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:31.561472 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.561461 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:31.563278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.563246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" event={"ID":"433f7dcb-b079-4507-bb70-854fe8fdeaf4","Type":"ContainerStarted","Data":"9c2afea8142ebd645503f9fd2a5e41a000c7324b5d718e14da2bb6f919bc57cb"} Jul 24 17:43:31.565009 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.564988 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="1924fde9725953760c0a150caa82ea094bffe7286cab03911e14f3d2d9196783" exitCode=0 Jul 24 17:43:31.565097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.565073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"1924fde9725953760c0a150caa82ea094bffe7286cab03911e14f3d2d9196783"} Jul 24 17:43:31.577266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.577247 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:31.599531 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.599469 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" podStartSLOduration=7.045573524 podStartE2EDuration="27.599454076s" totalImagesPullingTime="20.553880552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:31.59909888 +0000 UTC m=+27.800331299" watchObservedRunningTime="2026-07-24 17:43:31.599454076 +0000 UTC m=+27.800686465" Jul 24 17:43:31.645483 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:31.645434 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lpvm5" podStartSLOduration=2.346285802 podStartE2EDuration="26.645413421s" totalImagesPullingTime="24.299127619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:31.645346434 +0000 UTC m=+27.846578836" watchObservedRunningTime="2026-07-24 17:43:31.645413421 +0000 UTC m=+27.846645810" Jul 24 17:43:32.243191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:32.243029 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:32.243370 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:32.243274 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:32.567920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:32.567821 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:32.583065 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:32.582779 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:43:33.243679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.243602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:33.243847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.243602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:33.243847 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:33.243720 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:33.243847 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:33.243767 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:33.571978 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.571886 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="05195896116e491850051af56b4d8f36be5f48a9c65c2fb95afb801e06e3bb5e" exitCode=0 Jul 24 17:43:33.571978 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.571966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"05195896116e491850051af56b4d8f36be5f48a9c65c2fb95afb801e06e3bb5e"} Jul 24 17:43:33.660257 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.660218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qdzsl"] Jul 24 17:43:33.660441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.660425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:33.660575 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:33.660541 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:33.663262 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.663236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6khh"] Jul 24 17:43:33.663388 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.663348 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:33.663476 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:33.663455 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:33.678235 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.678207 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrnmv"] Jul 24 17:43:33.678399 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:33.678317 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:33.678473 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:33.678445 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:35.243465 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:35.243428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:35.243899 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:35.243434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:35.243899 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:35.243660 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:35.243899 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:35.243435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:35.243899 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:35.243532 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:35.243899 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:35.243773 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:35.578374 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:35.578280 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="d31f3dd32e6ee3eaf2efaff98987fc4750842816f635cc7056c4b30c4b1f4e84" exitCode=0 Jul 24 17:43:35.578374 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:35.578323 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"d31f3dd32e6ee3eaf2efaff98987fc4750842816f635cc7056c4b30c4b1f4e84"} Jul 24 17:43:36.889278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:36.889242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:36.889779 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.889396 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 17:43:36.889779 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.889413 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 17:43:36.889779 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.889422 2566 projected.go:196] Error preparing data for projected volume kube-api-access-smt82 for pod openshift-network-diagnostics/network-check-target-zrnmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:36.889779 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.889473 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82 podName:668e5331-d82e-4d6a-8df5-bf8e6a033949 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:08.889458287 +0000 UTC m=+65.090690654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-smt82" (UniqueName: "kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82") pod "network-check-target-zrnmv" (UID: "668e5331-d82e-4d6a-8df5-bf8e6a033949") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 17:43:36.990376 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:36.990328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:36.990536 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:36.990387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:36.990536 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.990483 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:36.990663 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.990551 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs podName:6cc1502b-f19f-416f-94ef-04e577a87b67 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:08.990529366 +0000 UTC m=+65.191761734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs") pod "network-metrics-daemon-b6khh" (UID: "6cc1502b-f19f-416f-94ef-04e577a87b67") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 17:43:36.990663 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.990484 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:36.990663 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:36.990637 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret podName:d6e14a3c-14e7-472b-9452-5b8f01744623 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:08.990616643 +0000 UTC m=+65.191849016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret") pod "global-pull-secret-syncer-qdzsl" (UID: "d6e14a3c-14e7-472b-9452-5b8f01744623") : object "kube-system"/"original-pull-secret" not registered Jul 24 17:43:37.243426 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:37.243330 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:37.243426 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:37.243394 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:37.243626 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:37.243339 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:37.243626 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:37.243472 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:37.243626 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:37.243527 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:37.243626 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:37.243606 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:39.243658 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.243621 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:39.244138 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.243731 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:39.244138 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.243744 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:39.244138 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:39.243742 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qdzsl" podUID="d6e14a3c-14e7-472b-9452-5b8f01744623" Jul 24 17:43:39.244138 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:39.243805 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrnmv" podUID="668e5331-d82e-4d6a-8df5-bf8e6a033949" Jul 24 17:43:39.244138 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:39.243886 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6khh" podUID="6cc1502b-f19f-416f-94ef-04e577a87b67" Jul 24 17:43:39.591085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.591047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerStarted","Data":"3914ca03a4406a5d45e14fa59b4ab9abcc295a0ad45298d5a41d7f86e9ba259a"} Jul 24 17:43:39.638564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.638543 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-71.ec2.internal" event="NodeReady" Jul 24 17:43:39.638675 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.638666 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 17:43:39.701227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.701196 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8kf99"] Jul 24 17:43:39.703031 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.703010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.703151 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.703036 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mjbzn"] Jul 24 17:43:39.705000 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.704983 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.707420 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.707401 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 17:43:39.707533 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.707454 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 17:43:39.707533 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.707468 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t2cd6\"" Jul 24 17:43:39.707533 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.707453 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 17:43:39.707842 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.707825 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 17:43:39.708183 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.708169 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 17:43:39.708272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.708257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5kpm4\"" Jul 24 17:43:39.721245 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.721223 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kf99"] Jul 24 17:43:39.729598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.729577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjbzn"] Jul 24 17:43:39.811456 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d32485a-b722-4c1e-8024-22ca8fd832c3-cert\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.811456 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wv9z\" (UniqueName: \"kubernetes.io/projected/0d32485a-b722-4c1e-8024-22ca8fd832c3-kube-api-access-6wv9z\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.811634 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811465 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6762dfe6-30a5-448c-afea-86fffc3ae2c7-tmp-dir\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.811634 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811506 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjhh\" (UniqueName: \"kubernetes.io/projected/6762dfe6-30a5-448c-afea-86fffc3ae2c7-kube-api-access-rvjhh\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.811634 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6762dfe6-30a5-448c-afea-86fffc3ae2c7-config-volume\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.811634 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.811546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.912677 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6762dfe6-30a5-448c-afea-86fffc3ae2c7-config-volume\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.912677 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.912847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d32485a-b722-4c1e-8024-22ca8fd832c3-cert\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.912847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wv9z\" (UniqueName: \"kubernetes.io/projected/0d32485a-b722-4c1e-8024-22ca8fd832c3-kube-api-access-6wv9z\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.912847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6762dfe6-30a5-448c-afea-86fffc3ae2c7-tmp-dir\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.912847 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:39.912823 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:39.912847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.912838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjhh\" (UniqueName: \"kubernetes.io/projected/6762dfe6-30a5-448c-afea-86fffc3ae2c7-kube-api-access-rvjhh\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.913071 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:39.912884 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:40.412866418 +0000 UTC m=+36.614098811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:39.913123 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.913086 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6762dfe6-30a5-448c-afea-86fffc3ae2c7-tmp-dir\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.913242 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.913224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6762dfe6-30a5-448c-afea-86fffc3ae2c7-config-volume\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.916901 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.916876 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d32485a-b722-4c1e-8024-22ca8fd832c3-cert\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:39.922391 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.922351 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjhh\" (UniqueName: \"kubernetes.io/projected/6762dfe6-30a5-448c-afea-86fffc3ae2c7-kube-api-access-rvjhh\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:39.922647 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:39.922629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wv9z\" (UniqueName: \"kubernetes.io/projected/0d32485a-b722-4c1e-8024-22ca8fd832c3-kube-api-access-6wv9z\") pod \"ingress-canary-8kf99\" (UID: \"0d32485a-b722-4c1e-8024-22ca8fd832c3\") " pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:40.013598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.013549 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kf99" Jul 24 17:43:40.145263 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.145089 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kf99"] Jul 24 17:43:40.148687 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:40.148655 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d32485a_b722_4c1e_8024_22ca8fd832c3.slice/crio-05709f7558601754f2f9009109f0675e0b0ba6814d8f521e4983b8f7efa276c2 WatchSource:0}: Error finding container 05709f7558601754f2f9009109f0675e0b0ba6814d8f521e4983b8f7efa276c2: Status 404 returned error can't find the container with id 05709f7558601754f2f9009109f0675e0b0ba6814d8f521e4983b8f7efa276c2 Jul 24 17:43:40.416800 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.416669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:40.417404 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:40.416827 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:40.417404 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:40.416892 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:41.416876047 +0000 UTC m=+37.618108414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:40.594941 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.594901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kf99" event={"ID":"0d32485a-b722-4c1e-8024-22ca8fd832c3","Type":"ContainerStarted","Data":"05709f7558601754f2f9009109f0675e0b0ba6814d8f521e4983b8f7efa276c2"} Jul 24 17:43:40.597616 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.597589 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="3914ca03a4406a5d45e14fa59b4ab9abcc295a0ad45298d5a41d7f86e9ba259a" exitCode=0 Jul 24 17:43:40.597770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:40.597654 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"3914ca03a4406a5d45e14fa59b4ab9abcc295a0ad45298d5a41d7f86e9ba259a"} Jul 24 17:43:41.242880 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.242847 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:43:41.243055 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.242847 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:43:41.243055 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.242847 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:43:41.246079 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.245920 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 17:43:41.246079 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.245993 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 17:43:41.247065 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.247035 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9txdv\"" Jul 24 17:43:41.247178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.247098 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdhgn\"" Jul 24 17:43:41.247178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.247163 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 17:43:41.247313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.247295 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 17:43:41.426916 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.426882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:41.427344 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:41.426996 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:41.427344 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:41.427055 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:43.427037322 +0000 UTC m=+39.628269703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:41.604387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.604289 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bb784c5-92b8-4ad5-9041-981977aa7cf3" containerID="2d4215ac7c7830aa3766309b99b31831eb5c50f562a736f573cb495f0667a713" exitCode=0 Jul 24 17:43:41.604387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:41.604340 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerDied","Data":"2d4215ac7c7830aa3766309b99b31831eb5c50f562a736f573cb495f0667a713"} Jul 24 17:43:42.609961 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:42.609928 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" event={"ID":"7bb784c5-92b8-4ad5-9041-981977aa7cf3","Type":"ContainerStarted","Data":"c3c6cfd53089d5b90c518b71eb6fa2388c193a1fc9a659ce696ba09f379bebbe"} Jul 24 17:43:42.641252 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:42.641040 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bq4cg" podStartSLOduration=7.215544004 podStartE2EDuration="37.641019597s" totalImagesPullingTime="30.425475593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:42.640484115 +0000 UTC m=+38.841716507" watchObservedRunningTime="2026-07-24 17:43:42.641019597 +0000 UTC m=+38.842251987" Jul 24 17:43:43.443439 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:43.443245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:43.443596 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:43.443411 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:43.443596 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:43.443558 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:47.443542235 +0000 UTC m=+43.644774603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:43.614060 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:43.614024 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kf99" event={"ID":"0d32485a-b722-4c1e-8024-22ca8fd832c3","Type":"ContainerStarted","Data":"ce0707f66ecb1a65bfd31607df7d11f8f512f39af84fc70ec2f8b832c13289b6"} Jul 24 17:43:43.638413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:43.638346 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8kf99" podStartSLOduration=2.292434001 podStartE2EDuration="4.638331203s" totalImagesPullingTime="2.345897202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:43.637976086 +0000 UTC m=+39.839208475" watchObservedRunningTime="2026-07-24 17:43:43.638331203 +0000 UTC m=+39.839563586" Jul 24 17:43:47.470815 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:47.470754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:47.471170 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:47.470937 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:47.471170 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:47.471001 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:55.470987401 +0000 UTC m=+51.672219768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:49.001993 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.001956 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sl8lt"] Jul 24 17:43:49.022201 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.022175 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sl8lt"] Jul 24 17:43:49.022339 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.022305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.024932 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.024916 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 17:43:49.025847 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.025831 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 17:43:49.025936 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.025863 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 17:43:49.025936 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.025879 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cg7vk\"" Jul 24 17:43:49.025936 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.025900 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 17:43:49.183561 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.183520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76r68\" (UniqueName: \"kubernetes.io/projected/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-api-access-76r68\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.183561 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.183563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5012c20-c068-4021-b3f9-f2ab33ffafa4-data-volume\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.183838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.183591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.183838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.183647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5012c20-c068-4021-b3f9-f2ab33ffafa4-crio-socket\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.183838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.183706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.284966 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.284865 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76r68\" (UniqueName: \"kubernetes.io/projected/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-api-access-76r68\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.284966 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.284911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5012c20-c068-4021-b3f9-f2ab33ffafa4-data-volume\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.284966 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.284943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.284973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5012c20-c068-4021-b3f9-f2ab33ffafa4-crio-socket\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.285023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.285045 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.285136 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:49.785114403 +0000 UTC m=+45.986346775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : configmap "kube-rbac-proxy" not found Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.285193 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:43:49.285258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.285201 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5012c20-c068-4021-b3f9-f2ab33ffafa4-crio-socket\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.285519 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.285273 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:49.785257284 +0000 UTC m=+45.986489659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:43:49.285519 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.285300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5012c20-c068-4021-b3f9-f2ab33ffafa4-data-volume\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.297710 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.297675 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76r68\" (UniqueName: \"kubernetes.io/projected/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-api-access-76r68\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.789703 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.789659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.789894 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:49.789787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:49.789894 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.789814 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:43:49.789894 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.789874 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:43:49.789894 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.789896 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:50.789878411 +0000 UTC m=+46.991110778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:43:49.790083 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:49.789968 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:50.78994999 +0000 UTC m=+46.991182361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : configmap "kube-rbac-proxy" not found Jul 24 17:43:50.297688 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.297657 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp"] Jul 24 17:43:50.330380 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.330325 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp"] Jul 24 17:43:50.330558 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.330460 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" Jul 24 17:43:50.333387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.333368 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 17:43:50.334378 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.334310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p2jnz\"" Jul 24 17:43:50.334378 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.334345 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 17:43:50.495665 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.495636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tk4m\" (UniqueName: \"kubernetes.io/projected/df2dd3b2-bfcb-4691-922d-8eacf64922f8-kube-api-access-8tk4m\") pod \"migrator-6865c9967-r9jtp\" (UID: \"df2dd3b2-bfcb-4691-922d-8eacf64922f8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" Jul 24 17:43:50.596903 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.596836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tk4m\" (UniqueName: \"kubernetes.io/projected/df2dd3b2-bfcb-4691-922d-8eacf64922f8-kube-api-access-8tk4m\") pod \"migrator-6865c9967-r9jtp\" (UID: \"df2dd3b2-bfcb-4691-922d-8eacf64922f8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" Jul 24 17:43:50.606034 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.606000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tk4m\" (UniqueName: \"kubernetes.io/projected/df2dd3b2-bfcb-4691-922d-8eacf64922f8-kube-api-access-8tk4m\") pod \"migrator-6865c9967-r9jtp\" (UID: \"df2dd3b2-bfcb-4691-922d-8eacf64922f8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" Jul 24 17:43:50.639446 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.639417 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" Jul 24 17:43:50.782135 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.782107 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp"] Jul 24 17:43:50.785019 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:50.784991 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2dd3b2_bfcb_4691_922d_8eacf64922f8.slice/crio-d8fbd4c156346ccef823ba61d63eaead2e457c11c7e8bedee6f795324e094a81 WatchSource:0}: Error finding container d8fbd4c156346ccef823ba61d63eaead2e457c11c7e8bedee6f795324e094a81: Status 404 returned error can't find the container with id d8fbd4c156346ccef823ba61d63eaead2e457c11c7e8bedee6f795324e094a81 Jul 24 17:43:50.798595 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.798571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:50.798714 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:50.798634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:50.798714 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:50.798694 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:43:50.798837 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:50.798750 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:43:50.798837 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:50.798757 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:52.798739962 +0000 UTC m=+48.999972349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : configmap "kube-rbac-proxy" not found Jul 24 17:43:50.798837 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:50.798800 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:52.79878486 +0000 UTC m=+49.000017233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:43:51.259227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.259203 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t9nvd_11364763-f913-455b-9687-aa0a8bc528bc/dns-node-resolver/0.log" Jul 24 17:43:51.621614 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.621583 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ks7n2"] Jul 24 17:43:51.651186 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.651146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ks7n2"] Jul 24 17:43:51.651341 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.651195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" event={"ID":"df2dd3b2-bfcb-4691-922d-8eacf64922f8","Type":"ContainerStarted","Data":"d8fbd4c156346ccef823ba61d63eaead2e457c11c7e8bedee6f795324e094a81"} Jul 24 17:43:51.651341 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.651275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.654278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.654221 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 17:43:51.654278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.654242 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 17:43:51.654278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.654259 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h77gs\"" Jul 24 17:43:51.655221 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.655198 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 17:43:51.655627 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.655604 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 17:43:51.704810 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.704776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-cabundle\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.704810 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.704814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkbx\" (UniqueName: \"kubernetes.io/projected/507929f6-989e-4d66-9d89-6b3a1dcefdc2-kube-api-access-gvkbx\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.705041 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.704865 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-key\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.805734 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.805690 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-key\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.805921 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.805841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-cabundle\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.805921 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.805870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkbx\" (UniqueName: \"kubernetes.io/projected/507929f6-989e-4d66-9d89-6b3a1dcefdc2-kube-api-access-gvkbx\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.806755 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.806728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-cabundle\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.808716 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.808686 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/507929f6-989e-4d66-9d89-6b3a1dcefdc2-signing-key\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.817027 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.816999 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkbx\" (UniqueName: \"kubernetes.io/projected/507929f6-989e-4d66-9d89-6b3a1dcefdc2-kube-api-access-gvkbx\") pod \"service-ca-658d7b7c78-ks7n2\" (UID: \"507929f6-989e-4d66-9d89-6b3a1dcefdc2\") " pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:51.961984 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:51.961901 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" Jul 24 17:43:52.109856 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.109806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ks7n2"] Jul 24 17:43:52.233093 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:43:52.233023 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507929f6_989e_4d66_9d89_6b3a1dcefdc2.slice/crio-c50a4a948da02a9d810eaff1e2ed1d1ae9937b4f469fc85d9e5b008b9b5c49ca WatchSource:0}: Error finding container c50a4a948da02a9d810eaff1e2ed1d1ae9937b4f469fc85d9e5b008b9b5c49ca: Status 404 returned error can't find the container with id c50a4a948da02a9d810eaff1e2ed1d1ae9937b4f469fc85d9e5b008b9b5c49ca Jul 24 17:43:52.263280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.263261 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhwnj_3ea17a03-d149-4a7c-9f03-47f288bf3f3c/node-ca/0.log" Jul 24 17:43:52.640415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.640376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" event={"ID":"507929f6-989e-4d66-9d89-6b3a1dcefdc2","Type":"ContainerStarted","Data":"c50a4a948da02a9d810eaff1e2ed1d1ae9937b4f469fc85d9e5b008b9b5c49ca"} Jul 24 17:43:52.642051 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.642028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" event={"ID":"df2dd3b2-bfcb-4691-922d-8eacf64922f8","Type":"ContainerStarted","Data":"ce20d6f8c0c97d46c91b7b973547fbf9f5f52c5982a74fefd7338b466aaeffec"} Jul 24 17:43:52.642158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.642068 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" event={"ID":"df2dd3b2-bfcb-4691-922d-8eacf64922f8","Type":"ContainerStarted","Data":"541b9fbaf1607ce509ac2dc9bf416924b9745e14602e45fb8dae4f7e237db925"} Jul 24 17:43:52.660713 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.660663 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-r9jtp" podStartSLOduration=1.191955927 podStartE2EDuration="2.660650099s" totalImagesPullingTime="1.468694172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:52.660188285 +0000 UTC m=+48.861420674" watchObservedRunningTime="2026-07-24 17:43:52.660650099 +0000 UTC m=+48.861882485" Jul 24 17:43:52.661696 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.661676 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8kf99_0d32485a-b722-4c1e-8024-22ca8fd832c3/serve-healthcheck-canary/0.log" Jul 24 17:43:52.814655 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.814621 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:52.814808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:52.814739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:52.814808 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:52.814769 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:43:52.814919 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:52.814840 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:43:52.814919 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:52.814843 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:56.814822978 +0000 UTC m=+53.016055347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:43:52.814919 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:52.814908 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:43:56.814891369 +0000 UTC m=+53.016123742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : configmap "kube-rbac-proxy" not found Jul 24 17:43:54.648660 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:54.648623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" event={"ID":"507929f6-989e-4d66-9d89-6b3a1dcefdc2","Type":"ContainerStarted","Data":"c7fb6eaa038c9a60710398c3660ade6c1a25ef26e13f2bc4b2974916242054b2"} Jul 24 17:43:54.664413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:54.664312 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-ks7n2" podStartSLOduration=1.514086351 podStartE2EDuration="3.664295157s" totalImagesPullingTime="2.150208806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:43:54.664251204 +0000 UTC m=+50.865483593" watchObservedRunningTime="2026-07-24 17:43:54.664295157 +0000 UTC m=+50.865527546" Jul 24 17:43:55.538371 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:55.538321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:43:55.538533 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:55.538490 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 17:43:55.538609 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:55.538597 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls podName:6762dfe6-30a5-448c-afea-86fffc3ae2c7 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:11.538575993 +0000 UTC m=+67.739808365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls") pod "dns-default-mjbzn" (UID: "6762dfe6-30a5-448c-afea-86fffc3ae2c7") : secret "dns-default-metrics-tls" not found Jul 24 17:43:56.848867 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:56.848830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:56.849234 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:43:56.848947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:43:56.849234 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:56.848980 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:43:56.849234 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:56.849041 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 17:43:56.849234 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:56.849054 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:04.849034957 +0000 UTC m=+61.050267325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:43:56.849234 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:43:56.849092 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:04.849075886 +0000 UTC m=+61.050308259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : configmap "kube-rbac-proxy" not found Jul 24 17:44:04.589416 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:04.589386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4n45h" Jul 24 17:44:04.905795 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:04.905762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:04.905953 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:04.905813 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:04.905953 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:04.905894 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 17:44:04.905953 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:04.905955 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls podName:c5012c20-c068-4021-b3f9-f2ab33ffafa4 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:20.905940272 +0000 UTC m=+77.107172638 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sl8lt" (UID: "c5012c20-c068-4021-b3f9-f2ab33ffafa4") : secret "insights-runtime-extractor-tls" not found Jul 24 17:44:04.906232 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:04.906211 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5012c20-c068-4021-b3f9-f2ab33ffafa4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:08.934252 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:08.934209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:44:08.936907 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:08.936888 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 17:44:08.947151 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:08.947123 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 17:44:08.957750 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:08.957718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smt82\" (UniqueName: \"kubernetes.io/projected/668e5331-d82e-4d6a-8df5-bf8e6a033949-kube-api-access-smt82\") pod \"network-check-target-zrnmv\" (UID: \"668e5331-d82e-4d6a-8df5-bf8e6a033949\") " pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:44:09.034943 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.034890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:44:09.034943 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.034952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:44:09.037998 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.037979 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 17:44:09.041412 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.041391 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 17:44:09.047993 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.047962 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cc1502b-f19f-416f-94ef-04e577a87b67-metrics-certs\") pod \"network-metrics-daemon-b6khh\" (UID: \"6cc1502b-f19f-416f-94ef-04e577a87b67\") " pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:44:09.047993 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.047967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6e14a3c-14e7-472b-9452-5b8f01744623-original-pull-secret\") pod \"global-pull-secret-syncer-qdzsl\" (UID: \"d6e14a3c-14e7-472b-9452-5b8f01744623\") " pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:44:09.156307 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.156270 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pdhgn\"" Jul 24 17:44:09.161461 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.161444 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qdzsl" Jul 24 17:44:09.164130 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.164110 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6khh" Jul 24 17:44:09.169381 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.169346 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9txdv\"" Jul 24 17:44:09.177525 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.177502 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:44:09.304788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.304758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qdzsl"] Jul 24 17:44:09.308965 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:09.308921 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e14a3c_14e7_472b_9452_5b8f01744623.slice/crio-e2712fda0235f08c43f0beeb3b4e82925a0d4f4f1ddb66398afe2fc53aa9dc57 WatchSource:0}: Error finding container e2712fda0235f08c43f0beeb3b4e82925a0d4f4f1ddb66398afe2fc53aa9dc57: Status 404 returned error can't find the container with id e2712fda0235f08c43f0beeb3b4e82925a0d4f4f1ddb66398afe2fc53aa9dc57 Jul 24 17:44:09.322174 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.322139 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6khh"] Jul 24 17:44:09.325257 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:09.325234 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc1502b_f19f_416f_94ef_04e577a87b67.slice/crio-df45c8f9d1a12e6d086a99bac7538ae3cef7f400b5d9ddd927d8388efbb69253 WatchSource:0}: Error finding container df45c8f9d1a12e6d086a99bac7538ae3cef7f400b5d9ddd927d8388efbb69253: Status 404 returned error can't find the container with id df45c8f9d1a12e6d086a99bac7538ae3cef7f400b5d9ddd927d8388efbb69253 Jul 24 17:44:09.345747 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.345724 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrnmv"] Jul 24 17:44:09.348468 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:09.348449 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668e5331_d82e_4d6a_8df5_bf8e6a033949.slice/crio-d783f09787cd7a028b97b2a2d8611708b6dea06fbbb1406a3b73cc8729ad6747 WatchSource:0}: Error finding container d783f09787cd7a028b97b2a2d8611708b6dea06fbbb1406a3b73cc8729ad6747: Status 404 returned error can't find the container with id d783f09787cd7a028b97b2a2d8611708b6dea06fbbb1406a3b73cc8729ad6747 Jul 24 17:44:09.684817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.684755 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrnmv" event={"ID":"668e5331-d82e-4d6a-8df5-bf8e6a033949","Type":"ContainerStarted","Data":"d783f09787cd7a028b97b2a2d8611708b6dea06fbbb1406a3b73cc8729ad6747"} Jul 24 17:44:09.686707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.686680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6khh" event={"ID":"6cc1502b-f19f-416f-94ef-04e577a87b67","Type":"ContainerStarted","Data":"df45c8f9d1a12e6d086a99bac7538ae3cef7f400b5d9ddd927d8388efbb69253"} Jul 24 17:44:09.687727 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:09.687703 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qdzsl" event={"ID":"d6e14a3c-14e7-472b-9452-5b8f01744623","Type":"ContainerStarted","Data":"e2712fda0235f08c43f0beeb3b4e82925a0d4f4f1ddb66398afe2fc53aa9dc57"} Jul 24 17:44:11.557517 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.557472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:44:11.560238 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.560192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6762dfe6-30a5-448c-afea-86fffc3ae2c7-metrics-tls\") pod \"dns-default-mjbzn\" (UID: \"6762dfe6-30a5-448c-afea-86fffc3ae2c7\") " pod="openshift-dns/dns-default-mjbzn" Jul 24 17:44:11.699504 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.699468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6khh" event={"ID":"6cc1502b-f19f-416f-94ef-04e577a87b67","Type":"ContainerStarted","Data":"6bda3f761823a4d30978724be50003491d58fef59d13ba8e0df023cab18ede78"} Jul 24 17:44:11.699669 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.699531 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6khh" event={"ID":"6cc1502b-f19f-416f-94ef-04e577a87b67","Type":"ContainerStarted","Data":"3216d7c62e614dd4a226dce71c0f101f63c0de084ff46ae6211cddf2eedeb6f4"} Jul 24 17:44:11.720325 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.719922 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b6khh" podStartSLOduration=65.414255926 podStartE2EDuration="1m6.719900909s" totalImagesPullingTime="1.305644983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:11.718778475 +0000 UTC m=+67.920010863" watchObservedRunningTime="2026-07-24 17:44:11.719900909 +0000 UTC m=+67.921133299" Jul 24 17:44:11.821085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.820991 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5kpm4\"" Jul 24 17:44:11.829675 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:11.829434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjbzn" Jul 24 17:44:14.129676 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.129645 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjbzn"] Jul 24 17:44:14.133629 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:14.133605 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6762dfe6_30a5_448c_afea_86fffc3ae2c7.slice/crio-6bb1947107eecdbe2469042fcaa0eb4f7267f5a88cb0e361cd9640bb348938a2 WatchSource:0}: Error finding container 6bb1947107eecdbe2469042fcaa0eb4f7267f5a88cb0e361cd9640bb348938a2: Status 404 returned error can't find the container with id 6bb1947107eecdbe2469042fcaa0eb4f7267f5a88cb0e361cd9640bb348938a2 Jul 24 17:44:14.711519 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.711473 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qdzsl" event={"ID":"d6e14a3c-14e7-472b-9452-5b8f01744623","Type":"ContainerStarted","Data":"d24bce4feb5a140d5393934f8bebf6deb331a34d05d9380eda1f29a0124d75fe"} Jul 24 17:44:14.713289 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.713261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrnmv" event={"ID":"668e5331-d82e-4d6a-8df5-bf8e6a033949","Type":"ContainerStarted","Data":"465be00106afa68e33ba91a7c4eed33d35303939a0fe2e7b412f8f9701a5e129"} Jul 24 17:44:14.713501 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.713462 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:44:14.714504 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.714478 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjbzn" event={"ID":"6762dfe6-30a5-448c-afea-86fffc3ae2c7","Type":"ContainerStarted","Data":"6bb1947107eecdbe2469042fcaa0eb4f7267f5a88cb0e361cd9640bb348938a2"} Jul 24 17:44:14.727691 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.727645 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qdzsl" podStartSLOduration=65.030206434 podStartE2EDuration="1m9.7276275s" totalImagesPullingTime="4.697421066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:14.727384837 +0000 UTC m=+70.928617254" watchObservedRunningTime="2026-07-24 17:44:14.7276275 +0000 UTC m=+70.928859888" Jul 24 17:44:14.743188 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:14.743136 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zrnmv" podStartSLOduration=66.087957469 podStartE2EDuration="1m10.743121911s" totalImagesPullingTime="4.655164442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:14.742202026 +0000 UTC m=+70.943434415" watchObservedRunningTime="2026-07-24 17:44:14.743121911 +0000 UTC m=+70.944354300" Jul 24 17:44:15.764752 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.764721 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:15.766301 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.766284 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.771136 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771117 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 17:44:15.771274 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771114 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 17:44:15.771274 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771143 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 17:44:15.771274 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771164 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 17:44:15.772283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771691 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nq87m\"" Jul 24 17:44:15.772283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771741 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 17:44:15.772283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.771795 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 17:44:15.772283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.772269 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 17:44:15.786716 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.786664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:15.879812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.879774 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77"] Jul 24 17:44:15.881669 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.881649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:15.885484 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.885462 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 24 17:44:15.885593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.885529 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-w4xsq\"" Jul 24 17:44:15.893603 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.893705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.893705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.893705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfjw6\" (UniqueName: \"kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.893705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.893705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.893692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.909932 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.909903 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77"] Jul 24 17:44:15.994671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a57443aa-74ee-4cd2-86f0-1f9a7de1d42a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-sjh77\" (UID: \"a57443aa-74ee-4cd2-86f0-1f9a7de1d42a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:15.994671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.994863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.994863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.994863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfjw6\" (UniqueName: \"kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.994863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.994863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.994852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.995569 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.995545 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.995569 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.995547 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.995788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.995606 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.997128 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.997109 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:15.997262 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:15.997245 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:16.003513 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.003491 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfjw6\" (UniqueName: \"kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6\") pod \"console-64cb96fb55-mtzpk\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:16.076110 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.076074 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:16.095491 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.095463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a57443aa-74ee-4cd2-86f0-1f9a7de1d42a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-sjh77\" (UID: \"a57443aa-74ee-4cd2-86f0-1f9a7de1d42a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:16.097938 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.097915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a57443aa-74ee-4cd2-86f0-1f9a7de1d42a-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-sjh77\" (UID: \"a57443aa-74ee-4cd2-86f0-1f9a7de1d42a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:16.190392 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.190346 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:16.209345 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.209315 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:16.212509 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:16.212483 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3ba038_7069_4ab3_86a6_832262b8531c.slice/crio-f82b19b8f3b491d5cda91e98f2627367535c152be99d59cc362e061d3ade02e7 WatchSource:0}: Error finding container f82b19b8f3b491d5cda91e98f2627367535c152be99d59cc362e061d3ade02e7: Status 404 returned error can't find the container with id f82b19b8f3b491d5cda91e98f2627367535c152be99d59cc362e061d3ade02e7 Jul 24 17:44:16.317303 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.317258 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77"] Jul 24 17:44:16.320552 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:16.320521 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57443aa_74ee_4cd2_86f0_1f9a7de1d42a.slice/crio-fe9c9545355c2579d76dc9277e43e5637532c2f88ae1f9e2e140d7688dbe79b2 WatchSource:0}: Error finding container fe9c9545355c2579d76dc9277e43e5637532c2f88ae1f9e2e140d7688dbe79b2: Status 404 returned error can't find the container with id fe9c9545355c2579d76dc9277e43e5637532c2f88ae1f9e2e140d7688dbe79b2 Jul 24 17:44:16.693651 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.693614 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:44:16.695672 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.695648 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.703719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.703696 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 17:44:16.709346 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.709317 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:44:16.724722 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.724689 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" event={"ID":"a57443aa-74ee-4cd2-86f0-1f9a7de1d42a","Type":"ContainerStarted","Data":"fe9c9545355c2579d76dc9277e43e5637532c2f88ae1f9e2e140d7688dbe79b2"} Jul 24 17:44:16.725876 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.725850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb96fb55-mtzpk" event={"ID":"4f3ba038-7069-4ab3-86a6-832262b8531c","Type":"ContainerStarted","Data":"f82b19b8f3b491d5cda91e98f2627367535c152be99d59cc362e061d3ade02e7"} Jul 24 17:44:16.727823 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.727795 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjbzn" event={"ID":"6762dfe6-30a5-448c-afea-86fffc3ae2c7","Type":"ContainerStarted","Data":"7fd62067c0180dd25cb7f58a84b4ddbc7e66dd406c38c808a6530d819fdc02ec"} Jul 24 17:44:16.727912 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.727846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjbzn" event={"ID":"6762dfe6-30a5-448c-afea-86fffc3ae2c7","Type":"ContainerStarted","Data":"3f11f296bb6bb99e856a504e3469fb3dd88a1bdc463a6ec4aded0b83897c17ae"} Jul 24 17:44:16.727975 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.727957 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mjbzn" Jul 24 17:44:16.746238 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.746177 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mjbzn" podStartSLOduration=36.193450905 podStartE2EDuration="37.746160443s" totalImagesPullingTime="1.552709538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:43:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:16.74557203 +0000 UTC m=+72.946804446" watchObservedRunningTime="2026-07-24 17:44:16.746160443 +0000 UTC m=+72.947392826" Jul 24 17:44:16.800780 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.800740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.800854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.800902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.800979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.801047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.801074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trx9r\" (UniqueName: \"kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.801197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.801158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.901977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trx9r\" (UniqueName: \"kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.902387 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.902023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.903788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.903167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.903788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.903201 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.903788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.903243 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.903788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.903743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.907828 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.907302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.907828 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.907775 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:16.914314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:16.914272 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trx9r\" (UniqueName: \"kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r\") pod \"console-c9bd57ccc-6vlhr\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:17.015898 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.015801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:17.192383 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.192304 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:44:17.198218 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:17.198178 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd087bc8_b4d6_4b1a_8c7e_7f62a68b3b47.slice/crio-ac8afad993486d984caea6509ae3230e68fb4ef5bd78c28cae5c912138fc613e WatchSource:0}: Error finding container ac8afad993486d984caea6509ae3230e68fb4ef5bd78c28cae5c912138fc613e: Status 404 returned error can't find the container with id ac8afad993486d984caea6509ae3230e68fb4ef5bd78c28cae5c912138fc613e Jul 24 17:44:17.733683 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.733641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" event={"ID":"a57443aa-74ee-4cd2-86f0-1f9a7de1d42a","Type":"ContainerStarted","Data":"035574e91736267521ce2196f80360bda92d75f8ff54745ad64d7780ca361b30"} Jul 24 17:44:17.734138 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.734120 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:17.735894 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.735504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bd57ccc-6vlhr" event={"ID":"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47","Type":"ContainerStarted","Data":"ac8afad993486d984caea6509ae3230e68fb4ef5bd78c28cae5c912138fc613e"} Jul 24 17:44:17.740627 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.740597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" Jul 24 17:44:17.751977 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:17.751640 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-sjh77" podStartSLOduration=1.530465814 podStartE2EDuration="2.751618581s" totalImagesPullingTime="1.221152767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:17.750551368 +0000 UTC m=+73.951783757" watchObservedRunningTime="2026-07-24 17:44:17.751618581 +0000 UTC m=+73.952850973" Jul 24 17:44:19.744522 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:19.744482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bd57ccc-6vlhr" event={"ID":"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47","Type":"ContainerStarted","Data":"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a"} Jul 24 17:44:19.746123 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:19.746095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb96fb55-mtzpk" event={"ID":"4f3ba038-7069-4ab3-86a6-832262b8531c","Type":"ContainerStarted","Data":"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61"} Jul 24 17:44:19.764294 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:19.764188 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9bd57ccc-6vlhr" podStartSLOduration=1.4513754620000001 podStartE2EDuration="3.764170708s" totalImagesPullingTime="2.312795246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:19.762323351 +0000 UTC m=+75.963555742" watchObservedRunningTime="2026-07-24 17:44:19.764170708 +0000 UTC m=+75.965403146" Jul 24 17:44:19.781629 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:19.781561 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64cb96fb55-mtzpk" podStartSLOduration=1.488733791 podStartE2EDuration="4.78154461s" totalImagesPullingTime="3.292810819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:19.779955547 +0000 UTC m=+75.981187948" watchObservedRunningTime="2026-07-24 17:44:19.78154461 +0000 UTC m=+75.982776998" Jul 24 17:44:20.937029 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:20.936987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:20.939250 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:20.939215 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5012c20-c068-4021-b3f9-f2ab33ffafa4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sl8lt\" (UID: \"c5012c20-c068-4021-b3f9-f2ab33ffafa4\") " pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:21.133985 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:21.133950 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cg7vk\"" Jul 24 17:44:21.142113 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:21.142093 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sl8lt" Jul 24 17:44:21.271297 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:21.271255 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sl8lt"] Jul 24 17:44:21.274228 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:21.274198 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5012c20_c068_4021_b3f9_f2ab33ffafa4.slice/crio-b452b6db133e1a7ff4788de40d6ad310d1ab0700819e348739e9863c5c6dfbb2 WatchSource:0}: Error finding container b452b6db133e1a7ff4788de40d6ad310d1ab0700819e348739e9863c5c6dfbb2: Status 404 returned error can't find the container with id b452b6db133e1a7ff4788de40d6ad310d1ab0700819e348739e9863c5c6dfbb2 Jul 24 17:44:21.756186 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:21.756148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sl8lt" event={"ID":"c5012c20-c068-4021-b3f9-f2ab33ffafa4","Type":"ContainerStarted","Data":"b452b6db133e1a7ff4788de40d6ad310d1ab0700819e348739e9863c5c6dfbb2"} Jul 24 17:44:22.804401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.801653 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l"] Jul 24 17:44:22.806817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.805439 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:22.810128 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.809919 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sjxrk\"" Jul 24 17:44:22.810128 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.810115 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 17:44:22.810309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.810288 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 17:44:22.810398 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.810374 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 17:44:22.810562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.810544 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 17:44:22.810624 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.810491 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 17:44:22.823008 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.822948 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs"] Jul 24 17:44:22.826106 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.826069 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l"] Jul 24 17:44:22.826253 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.826239 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.829841 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.829820 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 17:44:22.830409 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.829997 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 17:44:22.830704 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.830639 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t6tc7\"" Jul 24 17:44:22.831032 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.831015 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 17:44:22.832577 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.832554 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-68c8z"] Jul 24 17:44:22.835555 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.835537 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.837399 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.837381 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs"] Jul 24 17:44:22.839061 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.839042 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 17:44:22.839481 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.839463 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 17:44:22.839829 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.839812 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 17:44:22.840233 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.840215 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mfdvq\"" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-accelerators-collector-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-sys\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-metrics-client-ca\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw2jf\" (UniqueName: \"kubernetes.io/projected/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-api-access-xw2jf\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/336afd0d-d237-415b-89ab-c8474111be70-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9dm\" (UniqueName: \"kubernetes.io/projected/288a623b-c0b6-41f6-a147-fb70fb82bf71-kube-api-access-dp9dm\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956579 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-root\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-textfile\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-wtmp\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:22.957085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szrnv\" (UniqueName: \"kubernetes.io/projected/336afd0d-d237-415b-89ab-c8474111be70-kube-api-access-szrnv\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:22.957889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956872 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:22.957889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:22.957889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:22.956928 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-textfile\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058169 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-wtmp\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szrnv\" (UniqueName: \"kubernetes.io/projected/336afd0d-d237-415b-89ab-c8474111be70-kube-api-access-szrnv\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-accelerators-collector-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.058486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.058411 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-wtmp\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:23.058792 2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:23.058867 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls podName:3eb63ec5-cbdb-40c9-944d-41fb7225e00f nodeName:}" failed. No retries permitted until 2026-07-24 17:44:23.55884548 +0000 UTC m=+79.760077853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-w99rs" (UID: "3eb63ec5-cbdb-40c9-944d-41fb7225e00f") : secret "kube-state-metrics-tls" not found Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-sys\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-metrics-client-ca\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059166 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw2jf\" (UniqueName: \"kubernetes.io/projected/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-api-access-xw2jf\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/336afd0d-d237-415b-89ab-c8474111be70-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059266 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9dm\" (UniqueName: \"kubernetes.io/projected/288a623b-c0b6-41f6-a147-fb70fb82bf71-kube-api-access-dp9dm\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-root\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059393 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-root\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059327 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-textfile\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/288a623b-c0b6-41f6-a147-fb70fb82bf71-sys\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.059843 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:23.059546 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 17:44:23.059843 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:23.059597 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls podName:288a623b-c0b6-41f6-a147-fb70fb82bf71 nodeName:}" failed. No retries permitted until 2026-07-24 17:44:23.559579035 +0000 UTC m=+79.760811408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls") pod "node-exporter-68c8z" (UID: "288a623b-c0b6-41f6-a147-fb70fb82bf71") : secret "node-exporter-tls" not found Jul 24 17:44:23.059843 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.059756 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-accelerators-collector-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.060671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.060252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/288a623b-c0b6-41f6-a147-fb70fb82bf71-metrics-client-ca\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.060671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.060384 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.060671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.060580 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.060671 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.060588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/336afd0d-d237-415b-89ab-c8474111be70-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.063462 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.063407 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.063979 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.063952 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.064496 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.064473 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.065984 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.065913 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/336afd0d-d237-415b-89ab-c8474111be70-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.067097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.067051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.074675 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.074622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9dm\" (UniqueName: \"kubernetes.io/projected/288a623b-c0b6-41f6-a147-fb70fb82bf71-kube-api-access-dp9dm\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.075925 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.075881 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw2jf\" (UniqueName: \"kubernetes.io/projected/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-api-access-xw2jf\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.077961 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.077924 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szrnv\" (UniqueName: \"kubernetes.io/projected/336afd0d-d237-415b-89ab-c8474111be70-kube-api-access-szrnv\") pod \"openshift-state-metrics-b8f49f5c6-2fp4l\" (UID: \"336afd0d-d237-415b-89ab-c8474111be70\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.122508 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.122312 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" Jul 24 17:44:23.287186 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.287090 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l"] Jul 24 17:44:23.290411 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:23.290380 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336afd0d_d237_415b_89ab_c8474111be70.slice/crio-3bca77562ea57331330c5f28cfcbaff437adb4fef3a16a8e085d9de834e12dc7 WatchSource:0}: Error finding container 3bca77562ea57331330c5f28cfcbaff437adb4fef3a16a8e085d9de834e12dc7: Status 404 returned error can't find the container with id 3bca77562ea57331330c5f28cfcbaff437adb4fef3a16a8e085d9de834e12dc7 Jul 24 17:44:23.563225 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.563186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.563225 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.563226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.565790 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.565764 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3eb63ec5-cbdb-40c9-944d-41fb7225e00f-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-w99rs\" (UID: \"3eb63ec5-cbdb-40c9-944d-41fb7225e00f\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.565894 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.565877 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/288a623b-c0b6-41f6-a147-fb70fb82bf71-node-exporter-tls\") pod \"node-exporter-68c8z\" (UID: \"288a623b-c0b6-41f6-a147-fb70fb82bf71\") " pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.748227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.748198 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" Jul 24 17:44:23.756929 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.756896 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68c8z" Jul 24 17:44:23.768082 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.767879 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" event={"ID":"336afd0d-d237-415b-89ab-c8474111be70","Type":"ContainerStarted","Data":"87814a7b90d2f5b721cbf2bf8cd05cec733a43f9378cd68e2a4b7f3ad6298c97"} Jul 24 17:44:23.768082 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.767955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" event={"ID":"336afd0d-d237-415b-89ab-c8474111be70","Type":"ContainerStarted","Data":"e8f2ce90709ab3cc841c630cc094adb9b68e9be6fd6c2eeda2991823a43c2a28"} Jul 24 17:44:23.768082 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.768007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" event={"ID":"336afd0d-d237-415b-89ab-c8474111be70","Type":"ContainerStarted","Data":"3bca77562ea57331330c5f28cfcbaff437adb4fef3a16a8e085d9de834e12dc7"} Jul 24 17:44:23.907559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.907519 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:44:23.913762 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.913729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:23.916413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916226 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 17:44:23.916413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916239 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 17:44:23.916413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916391 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 17:44:23.916734 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916519 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 17:44:23.916734 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916604 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9z47g\"" Jul 24 17:44:23.916848 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916752 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 17:44:23.916848 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916790 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 17:44:23.916848 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916842 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 17:44:23.916988 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916910 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 17:44:23.916988 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.916964 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 17:44:23.925427 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:23.925402 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:44:24.068496 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068496 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068874 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068874 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068874 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvjrz\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068977 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068977 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.068977 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.068968 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.069171 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.069008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.169964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.169917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.169996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvjrz\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.170996 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.170974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.171145 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.171122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.171240 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.171222 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.173694 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.173572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.173694 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.173642 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.173931 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.173909 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.174259 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.174232 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.174725 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.174701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.174970 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.174919 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.176119 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.175579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.176207 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.176116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.177314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.177288 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.180662 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.180629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvjrz\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz\") pod \"alertmanager-main-0\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:24.227634 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:24.227601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:44:26.076854 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.076808 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:26.077306 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.076885 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:26.081992 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.081965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:26.459117 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:26.459086 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod288a623b_c0b6_41f6_a147_fb70fb82bf71.slice/crio-b6d766b53e8e0eeac5c9eb344a05f16a13c619d92354717eac473acc831eabd0 WatchSource:0}: Error finding container b6d766b53e8e0eeac5c9eb344a05f16a13c619d92354717eac473acc831eabd0: Status 404 returned error can't find the container with id b6d766b53e8e0eeac5c9eb344a05f16a13c619d92354717eac473acc831eabd0 Jul 24 17:44:26.597918 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.597806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs"] Jul 24 17:44:26.626989 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.626955 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:44:26.734858 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:26.734780 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb63ec5_cbdb_40c9_944d_41fb7225e00f.slice/crio-c0798ead49ea57922df79d0d1a48c95735af3505003f0cbe6e64445a28bbaf8f WatchSource:0}: Error finding container c0798ead49ea57922df79d0d1a48c95735af3505003f0cbe6e64445a28bbaf8f: Status 404 returned error can't find the container with id c0798ead49ea57922df79d0d1a48c95735af3505003f0cbe6e64445a28bbaf8f Jul 24 17:44:26.735509 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:26.735467 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabf74ed_dd52_4d49_be2c_9e6dd7636442.slice/crio-b1ff8b4d952a32e04b8cff1cf19287f96e3f6a404231b2417e66c6bdf5f5e8fa WatchSource:0}: Error finding container b1ff8b4d952a32e04b8cff1cf19287f96e3f6a404231b2417e66c6bdf5f5e8fa: Status 404 returned error can't find the container with id b1ff8b4d952a32e04b8cff1cf19287f96e3f6a404231b2417e66c6bdf5f5e8fa Jul 24 17:44:26.737836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.737817 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mjbzn" Jul 24 17:44:26.782887 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.782785 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" event={"ID":"3eb63ec5-cbdb-40c9-944d-41fb7225e00f","Type":"ContainerStarted","Data":"c0798ead49ea57922df79d0d1a48c95735af3505003f0cbe6e64445a28bbaf8f"} Jul 24 17:44:26.784214 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.784192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"b1ff8b4d952a32e04b8cff1cf19287f96e3f6a404231b2417e66c6bdf5f5e8fa"} Jul 24 17:44:26.785508 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.785481 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68c8z" event={"ID":"288a623b-c0b6-41f6-a147-fb70fb82bf71","Type":"ContainerStarted","Data":"b6d766b53e8e0eeac5c9eb344a05f16a13c619d92354717eac473acc831eabd0"} Jul 24 17:44:26.791198 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:26.791172 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:27.016526 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.016434 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:27.016683 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.016546 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:27.021107 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.021086 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:27.798515 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.797967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" event={"ID":"336afd0d-d237-415b-89ab-c8474111be70","Type":"ContainerStarted","Data":"abb47b76773612574d0f017072aea104927869cb8312b692096a6e2915f63e21"} Jul 24 17:44:27.803788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.802199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sl8lt" event={"ID":"c5012c20-c068-4021-b3f9-f2ab33ffafa4","Type":"ContainerStarted","Data":"411831cd283672b4891755c84569d368af0552c8d9cc2f42501a9bc957af6489"} Jul 24 17:44:27.812211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.810390 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:44:27.821006 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.820761 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-2fp4l" podStartSLOduration=2.480243331 podStartE2EDuration="5.820740473s" totalImagesPullingTime="3.340497142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:27.819682117 +0000 UTC m=+84.020914506" watchObservedRunningTime="2026-07-24 17:44:27.820740473 +0000 UTC m=+84.021972864" Jul 24 17:44:27.867798 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:27.867766 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:28.807663 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:28.807627 2566 generic.go:356] "Generic (PLEG): container finished" podID="288a623b-c0b6-41f6-a147-fb70fb82bf71" containerID="1e3317ab123445e254953c74f7193e2eadcb12cc274f81583598be7143f98412" exitCode=0 Jul 24 17:44:28.808113 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:28.807738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68c8z" event={"ID":"288a623b-c0b6-41f6-a147-fb70fb82bf71","Type":"ContainerDied","Data":"1e3317ab123445e254953c74f7193e2eadcb12cc274f81583598be7143f98412"} Jul 24 17:44:28.809970 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:28.809926 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" event={"ID":"3eb63ec5-cbdb-40c9-944d-41fb7225e00f","Type":"ContainerStarted","Data":"89ac901d4f7a4cd56958008b9ffe5e3c3f056c66b56b0d75f6a90059941e6978"} Jul 24 17:44:30.819083 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:30.819055 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5" exitCode=0 Jul 24 17:44:30.819376 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:30.819130 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5"} Jul 24 17:44:31.827486 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.827450 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68c8z" event={"ID":"288a623b-c0b6-41f6-a147-fb70fb82bf71","Type":"ContainerStarted","Data":"7815dd62f632994db31b19caac989812a1735be12974a72ee68186280ed276f3"} Jul 24 17:44:31.828036 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.827509 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68c8z" event={"ID":"288a623b-c0b6-41f6-a147-fb70fb82bf71","Type":"ContainerStarted","Data":"50de4ab09733fbd0e7b9d27e73349a582d0cb62ff5393f5eca34204aa56b44f4"} Jul 24 17:44:31.830454 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.830315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" event={"ID":"3eb63ec5-cbdb-40c9-944d-41fb7225e00f","Type":"ContainerStarted","Data":"dcbd99e9f7f4f1fd7909910722783342cde89d984a40feafbd7a38cf3119ac51"} Jul 24 17:44:31.830454 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.830407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" event={"ID":"3eb63ec5-cbdb-40c9-944d-41fb7225e00f","Type":"ContainerStarted","Data":"237a2cdbf9431e6b39eb97fea0d4a4eff7f5236df0db4f8446546408511e2081"} Jul 24 17:44:31.832550 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.832518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sl8lt" event={"ID":"c5012c20-c068-4021-b3f9-f2ab33ffafa4","Type":"ContainerStarted","Data":"7225cebcc6908e2e7e896783533009ebf4a05ec84398bc59ce2a93c9d2dac675"} Jul 24 17:44:31.852070 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.851408 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-68c8z" podStartSLOduration=8.321799649999999 podStartE2EDuration="9.851261328s" totalImagesPullingTime="1.417877375s" totalInitContainerRuntime="111.584303ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:31.849634203 +0000 UTC m=+88.050866623" watchObservedRunningTime="2026-07-24 17:44:31.851261328 +0000 UTC m=+88.052493718" Jul 24 17:44:31.875458 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:31.875346 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-w99rs" podStartSLOduration=7.924811115 podStartE2EDuration="9.875323977s" totalImagesPullingTime="1.950512862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:31.873123944 +0000 UTC m=+88.074356355" watchObservedRunningTime="2026-07-24 17:44:31.875323977 +0000 UTC m=+88.076556379" Jul 24 17:44:32.840302 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:32.839656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a"} Jul 24 17:44:32.840302 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:32.839786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96"} Jul 24 17:44:32.840302 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:32.839836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44"} Jul 24 17:44:32.840302 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:32.839877 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914"} Jul 24 17:44:34.849633 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:34.849541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sl8lt" event={"ID":"c5012c20-c068-4021-b3f9-f2ab33ffafa4","Type":"ContainerStarted","Data":"c1df4c7ee97e992d60329700759f0aeeda0420bd49550f7cec0b96d292cd7f45"} Jul 24 17:44:34.852598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:34.852573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05"} Jul 24 17:44:34.888235 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:34.888177 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sl8lt" podStartSLOduration=33.862246089 podStartE2EDuration="46.888160542s" totalImagesPullingTime="13.025914453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:43:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:34.887665874 +0000 UTC m=+91.088898264" watchObservedRunningTime="2026-07-24 17:44:34.888160542 +0000 UTC m=+91.089392930" Jul 24 17:44:35.703649 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.703564 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:44:35.708143 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.708104 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.720435 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.720408 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:44:35.859569 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.859535 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerStarted","Data":"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965"} Jul 24 17:44:35.877511 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9pn\" (UniqueName: \"kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877698 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877559 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877698 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877698 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877698 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877851 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877721 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.877851 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.877746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.891204 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.891148 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.406654428 podStartE2EDuration="12.891130904s" totalImagesPullingTime="4.464096147s" totalInitContainerRuntime="20.380329ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:35.889090675 +0000 UTC m=+92.090323105" watchObservedRunningTime="2026-07-24 17:44:35.891130904 +0000 UTC m=+92.092363338" Jul 24 17:44:35.979258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979193 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979680 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979597 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.979738 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.979717 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9pn\" (UniqueName: \"kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.980094 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.980067 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.980224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.980139 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.980418 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.980334 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.980856 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.980834 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.982606 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.982588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.982772 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.982751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:35.990165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:35.990140 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9pn\" (UniqueName: \"kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn\") pod \"console-9d8bbbf8c-v75xd\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:36.018947 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:36.018915 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:36.161481 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:36.161455 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:44:36.163623 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:36.163598 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8773feb9_5023_4e1b_9d23_ecb60801ba05.slice/crio-3e9ba38ee9a6e81309f314fb4b0fd395eeaaa00795cc979be95d98365209b63f WatchSource:0}: Error finding container 3e9ba38ee9a6e81309f314fb4b0fd395eeaaa00795cc979be95d98365209b63f: Status 404 returned error can't find the container with id 3e9ba38ee9a6e81309f314fb4b0fd395eeaaa00795cc979be95d98365209b63f Jul 24 17:44:36.865451 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:36.865411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d8bbbf8c-v75xd" event={"ID":"8773feb9-5023-4e1b-9d23-ecb60801ba05","Type":"ContainerStarted","Data":"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb"} Jul 24 17:44:36.865848 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:36.865472 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d8bbbf8c-v75xd" event={"ID":"8773feb9-5023-4e1b-9d23-ecb60801ba05","Type":"ContainerStarted","Data":"3e9ba38ee9a6e81309f314fb4b0fd395eeaaa00795cc979be95d98365209b63f"} Jul 24 17:44:36.883319 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:36.883259 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9d8bbbf8c-v75xd" podStartSLOduration=1.8832431490000001 podStartE2EDuration="1.883243149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:36.882012326 +0000 UTC m=+93.083244716" watchObservedRunningTime="2026-07-24 17:44:36.883243149 +0000 UTC m=+93.084475607" Jul 24 17:44:45.722056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:45.722024 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zrnmv" Jul 24 17:44:46.019809 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.019713 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:46.019809 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.019785 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:46.024483 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.024456 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:46.603770 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.603733 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:44:46.641271 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.641232 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:44:46.672813 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.672778 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:44:46.672963 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.672888 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769191 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769156 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dxcr\" (UniqueName: \"kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.769638 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.769464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.870388 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.870278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.870616 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.870598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.870787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.870773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.870900 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.870887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dxcr\" (UniqueName: \"kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.871004 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.870992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.871123 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.871112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.871222 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.871210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.874749 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.874518 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.874749 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.874706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.874749 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.874736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.875157 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.875134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.875921 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.875897 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.876482 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.876466 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.884244 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.884223 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dxcr\" (UniqueName: \"kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr\") pod \"console-64c8b575d7-4gz7x\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:46.906067 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.906041 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:44:46.982321 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:46.982285 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:47.158834 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:47.158757 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:44:47.161639 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:44:47.161615 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0ff36a_4d21_4dfc_976d_a58b2f1ec289.slice/crio-736b4d1dec726cf3673f3caf891741b31ad7e705c49a3d43343d31456d0244d0 WatchSource:0}: Error finding container 736b4d1dec726cf3673f3caf891741b31ad7e705c49a3d43343d31456d0244d0: Status 404 returned error can't find the container with id 736b4d1dec726cf3673f3caf891741b31ad7e705c49a3d43343d31456d0244d0 Jul 24 17:44:47.907403 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:47.907346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c8b575d7-4gz7x" event={"ID":"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289","Type":"ContainerStarted","Data":"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d"} Jul 24 17:44:47.907902 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:47.907421 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c8b575d7-4gz7x" event={"ID":"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289","Type":"ContainerStarted","Data":"736b4d1dec726cf3673f3caf891741b31ad7e705c49a3d43343d31456d0244d0"} Jul 24 17:44:47.930423 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:47.930347 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64c8b575d7-4gz7x" podStartSLOduration=1.930333372 podStartE2EDuration="1.930333372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:44:47.930065221 +0000 UTC m=+104.131297616" watchObservedRunningTime="2026-07-24 17:44:47.930333372 +0000 UTC m=+104.131565761" Jul 24 17:44:53.833622 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:53.833576 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64cb96fb55-mtzpk" podUID="4f3ba038-7069-4ab3-86a6-832262b8531c" containerName="console" containerID="cri-o://8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61" gracePeriod=15 Jul 24 17:44:54.078772 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.078747 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:54.135588 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135546 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.135588 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135593 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.135844 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135624 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.135844 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135753 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.136021 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135843 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.136021 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.135885 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfjw6\" (UniqueName: \"kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6\") pod \"4f3ba038-7069-4ab3-86a6-832262b8531c\" (UID: \"4f3ba038-7069-4ab3-86a6-832262b8531c\") " Jul 24 17:44:54.136159 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.136135 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:44:54.136221 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.136148 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:44:54.136280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.136234 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config" (OuterVolumeSpecName: "console-config") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:44:54.137933 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.137902 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:44:54.138061 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.137974 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:44:54.138061 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.138014 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6" (OuterVolumeSpecName: "kube-api-access-vfjw6") pod "4f3ba038-7069-4ab3-86a6-832262b8531c" (UID: "4f3ba038-7069-4ab3-86a6-832262b8531c"). InnerVolumeSpecName "kube-api-access-vfjw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:44:54.236586 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236551 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-console-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.236586 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236583 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfjw6\" (UniqueName: \"kubernetes.io/projected/4f3ba038-7069-4ab3-86a6-832262b8531c-kube-api-access-vfjw6\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.236840 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236596 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.236840 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236608 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-oauth-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.236840 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236619 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3ba038-7069-4ab3-86a6-832262b8531c-console-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.236840 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.236632 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f3ba038-7069-4ab3-86a6-832262b8531c-oauth-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:44:54.933926 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.933889 2566 generic.go:356] "Generic (PLEG): container finished" podID="4f3ba038-7069-4ab3-86a6-832262b8531c" containerID="8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61" exitCode=0 Jul 24 17:44:54.934335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.933951 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cb96fb55-mtzpk" Jul 24 17:44:54.934335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.933973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb96fb55-mtzpk" event={"ID":"4f3ba038-7069-4ab3-86a6-832262b8531c","Type":"ContainerDied","Data":"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61"} Jul 24 17:44:54.934335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.934050 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cb96fb55-mtzpk" event={"ID":"4f3ba038-7069-4ab3-86a6-832262b8531c","Type":"ContainerDied","Data":"f82b19b8f3b491d5cda91e98f2627367535c152be99d59cc362e061d3ade02e7"} Jul 24 17:44:54.934335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.934079 2566 scope.go:117] "RemoveContainer" containerID="8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61" Jul 24 17:44:54.942499 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.942477 2566 scope.go:117] "RemoveContainer" containerID="8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61" Jul 24 17:44:54.942810 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:44:54.942787 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61\": container with ID starting with 8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61 not found: ID does not exist" containerID="8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61" Jul 24 17:44:54.942878 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.942818 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61"} err="failed to get container status \"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61\": rpc error: code = NotFound desc = could not find container \"8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61\": container with ID starting with 8bc57365ea201c9f4f83da451e8a121fb7348f3313cfa2fc8638f24080a7bf61 not found: ID does not exist" Jul 24 17:44:54.950686 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.950658 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:54.954210 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:54.954184 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64cb96fb55-mtzpk"] Jul 24 17:44:56.247975 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:56.247931 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f3ba038-7069-4ab3-86a6-832262b8531c" path="/var/lib/kubelet/pods/4f3ba038-7069-4ab3-86a6-832262b8531c/volumes" Jul 24 17:44:56.983225 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:56.983187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:56.983445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:56.983256 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:56.988072 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:56.988049 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:57.949995 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:57.949968 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:44:57.994870 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:44:57.994839 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:45:12.925699 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:12.925646 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9d8bbbf8c-v75xd" podUID="8773feb9-5023-4e1b-9d23-ecb60801ba05" containerName="console" containerID="cri-o://8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb" gracePeriod=15 Jul 24 17:45:13.167398 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.167349 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:45:13.298786 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298696 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.298786 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298762 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx9pn\" (UniqueName: \"kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.298786 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298780 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.299049 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298838 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.299049 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298858 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.299049 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298882 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.299049 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.298924 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config\") pod \"8773feb9-5023-4e1b-9d23-ecb60801ba05\" (UID: \"8773feb9-5023-4e1b-9d23-ecb60801ba05\") " Jul 24 17:45:13.299458 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.299301 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca" (OuterVolumeSpecName: "service-ca") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:13.299458 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.299458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config" (OuterVolumeSpecName: "console-config") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:13.299650 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.299472 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:13.299650 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.299492 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:13.301010 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.300986 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:13.301376 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.301343 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:13.301443 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.301378 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn" (OuterVolumeSpecName: "kube-api-access-kx9pn") pod "8773feb9-5023-4e1b-9d23-ecb60801ba05" (UID: "8773feb9-5023-4e1b-9d23-ecb60801ba05"). InnerVolumeSpecName "kube-api-access-kx9pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:45:13.399920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399878 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.399920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399912 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.399920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399922 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx9pn\" (UniqueName: \"kubernetes.io/projected/8773feb9-5023-4e1b-9d23-ecb60801ba05-kube-api-access-kx9pn\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.399920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399931 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8773feb9-5023-4e1b-9d23-ecb60801ba05-console-oauth-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.400172 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399940 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.400172 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399948 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-oauth-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:13.400172 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:13.399955 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8773feb9-5023-4e1b-9d23-ecb60801ba05-trusted-ca-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:14.007565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.007525 2566 generic.go:356] "Generic (PLEG): container finished" podID="8773feb9-5023-4e1b-9d23-ecb60801ba05" containerID="8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb" exitCode=0 Jul 24 17:45:14.007945 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.007574 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d8bbbf8c-v75xd" Jul 24 17:45:14.007945 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.007615 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d8bbbf8c-v75xd" event={"ID":"8773feb9-5023-4e1b-9d23-ecb60801ba05","Type":"ContainerDied","Data":"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb"} Jul 24 17:45:14.007945 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.007669 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d8bbbf8c-v75xd" event={"ID":"8773feb9-5023-4e1b-9d23-ecb60801ba05","Type":"ContainerDied","Data":"3e9ba38ee9a6e81309f314fb4b0fd395eeaaa00795cc979be95d98365209b63f"} Jul 24 17:45:14.007945 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.007689 2566 scope.go:117] "RemoveContainer" containerID="8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb" Jul 24 17:45:14.016379 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.016335 2566 scope.go:117] "RemoveContainer" containerID="8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb" Jul 24 17:45:14.016637 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:14.016616 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb\": container with ID starting with 8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb not found: ID does not exist" containerID="8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb" Jul 24 17:45:14.016705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.016646 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb"} err="failed to get container status \"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb\": rpc error: code = NotFound desc = could not find container \"8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb\": container with ID starting with 8560ad17c4c89d265effd4034dc9fbb267576f2e6b39cdd2a5f162c95406d8cb not found: ID does not exist" Jul 24 17:45:14.029617 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.029585 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:45:14.033851 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.033825 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9d8bbbf8c-v75xd"] Jul 24 17:45:14.247925 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:14.247889 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8773feb9-5023-4e1b-9d23-ecb60801ba05" path="/var/lib/kubelet/pods/8773feb9-5023-4e1b-9d23-ecb60801ba05/volumes" Jul 24 17:45:23.017788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.017724 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c9bd57ccc-6vlhr" podUID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" containerName="console" containerID="cri-o://eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a" gracePeriod=15 Jul 24 17:45:23.251772 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.251748 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:45:23.383399 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383340 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383594 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383408 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383594 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383443 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383594 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383476 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383594 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383545 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383594 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383583 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383607 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trx9r\" (UniqueName: \"kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r\") pod \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\" (UID: \"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47\") " Jul 24 17:45:23.383885 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383832 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config" (OuterVolumeSpecName: "console-config") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:23.383885 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.383839 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:23.384197 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.384167 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca" (OuterVolumeSpecName: "service-ca") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:23.384316 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.384189 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:23.385652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.385627 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:23.385742 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.385707 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r" (OuterVolumeSpecName: "kube-api-access-trx9r") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "kube-api-access-trx9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:45:23.385794 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.385752 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" (UID: "dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:23.484518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484468 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-trusted-ca-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484511 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484523 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trx9r\" (UniqueName: \"kubernetes.io/projected/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-kube-api-access-trx9r\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484532 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-oauth-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484540 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484834 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484549 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-oauth-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:23.484834 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:23.484557 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47-console-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:24.046551 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.046516 2566 generic.go:356] "Generic (PLEG): container finished" podID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" containerID="eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a" exitCode=0 Jul 24 17:45:24.046940 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.046559 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bd57ccc-6vlhr" Jul 24 17:45:24.046940 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.046601 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bd57ccc-6vlhr" event={"ID":"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47","Type":"ContainerDied","Data":"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a"} Jul 24 17:45:24.046940 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.046651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bd57ccc-6vlhr" event={"ID":"dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47","Type":"ContainerDied","Data":"ac8afad993486d984caea6509ae3230e68fb4ef5bd78c28cae5c912138fc613e"} Jul 24 17:45:24.046940 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.046673 2566 scope.go:117] "RemoveContainer" containerID="eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a" Jul 24 17:45:24.057503 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.057482 2566 scope.go:117] "RemoveContainer" containerID="eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a" Jul 24 17:45:24.057743 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:24.057724 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a\": container with ID starting with eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a not found: ID does not exist" containerID="eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a" Jul 24 17:45:24.057791 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.057751 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a"} err="failed to get container status \"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a\": rpc error: code = NotFound desc = could not find container \"eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a\": container with ID starting with eccc4de0d37f213eed7843e71503da3ef68c2f080282972f7a736fecc51eeb8a not found: ID does not exist" Jul 24 17:45:24.069100 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.069075 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:45:24.075334 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.075312 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c9bd57ccc-6vlhr"] Jul 24 17:45:24.247887 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:24.247856 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" path="/var/lib/kubelet/pods/dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47/volumes" Jul 24 17:45:41.094343 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094303 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094622 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f3ba038-7069-4ab3-86a6-832262b8531c" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094636 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f3ba038-7069-4ab3-86a6-832262b8531c" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094645 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094650 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094659 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8773feb9-5023-4e1b-9d23-ecb60801ba05" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094665 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8773feb9-5023-4e1b-9d23-ecb60801ba05" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094716 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8773feb9-5023-4e1b-9d23-ecb60801ba05" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094724 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f3ba038-7069-4ab3-86a6-832262b8531c" containerName="console" Jul 24 17:45:41.094833 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.094729 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd087bc8-b4d6-4b1a-8c7e-7f62a68b3b47" containerName="console" Jul 24 17:45:41.100227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.100197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128095 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128059 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8kth\" (UniqueName: \"kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128244 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128244 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128244 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128165 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128244 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128418 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.128418 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.128303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.136751 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.136726 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:45:41.228870 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8kth\" (UniqueName: \"kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228929 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.228987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.229021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229781 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.229718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229781 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.229736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.229781 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.229718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.230026 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.230000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.231415 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.231383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.231519 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.231501 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.237991 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.237966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8kth\" (UniqueName: \"kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth\") pod \"console-6b69b97dff-4x4kq\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.410107 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.410077 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:41.563984 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:41.563949 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:45:41.567117 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:45:41.567088 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cafddd_3065_4dba_bc3a_e7f7a9ba3d1e.slice/crio-fa48a80a608c33ad3a9a39822752663d776fa1858914635f3793012e688c10c6 WatchSource:0}: Error finding container fa48a80a608c33ad3a9a39822752663d776fa1858914635f3793012e688c10c6: Status 404 returned error can't find the container with id fa48a80a608c33ad3a9a39822752663d776fa1858914635f3793012e688c10c6 Jul 24 17:45:42.119503 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:42.119464 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69b97dff-4x4kq" event={"ID":"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e","Type":"ContainerStarted","Data":"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86"} Jul 24 17:45:42.119890 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:42.119523 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69b97dff-4x4kq" event={"ID":"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e","Type":"ContainerStarted","Data":"fa48a80a608c33ad3a9a39822752663d776fa1858914635f3793012e688c10c6"} Jul 24 17:45:42.144677 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:42.144282 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b69b97dff-4x4kq" podStartSLOduration=1.144257982 podStartE2EDuration="1.144257982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:42.141659843 +0000 UTC m=+158.342892232" watchObservedRunningTime="2026-07-24 17:45:42.144257982 +0000 UTC m=+158.345490372" Jul 24 17:45:43.144765 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.144730 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:43.145175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145000 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="alertmanager" containerID="cri-o://c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" gracePeriod=120 Jul 24 17:45:43.145175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145066 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-metric" containerID="cri-o://df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" gracePeriod=120 Jul 24 17:45:43.145175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145098 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="prom-label-proxy" containerID="cri-o://b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" gracePeriod=120 Jul 24 17:45:43.145175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145134 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-web" containerID="cri-o://33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" gracePeriod=120 Jul 24 17:45:43.145175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145153 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="config-reloader" containerID="cri-o://4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" gracePeriod=120 Jul 24 17:45:43.145444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:43.145177 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy" containerID="cri-o://0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" gracePeriod=120 Jul 24 17:45:44.130042 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130011 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" exitCode=0 Jul 24 17:45:44.130042 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130034 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" exitCode=0 Jul 24 17:45:44.130042 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130040 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" exitCode=0 Jul 24 17:45:44.130042 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130046 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" exitCode=0 Jul 24 17:45:44.130309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130082 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965"} Jul 24 17:45:44.130309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a"} Jul 24 17:45:44.130309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44"} Jul 24 17:45:44.130309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.130194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914"} Jul 24 17:45:44.386861 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.386806 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:44.475050 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475019 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475079 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475109 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475136 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475160 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475228 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475208 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475241 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475276 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475300 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475345 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475394 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvjrz\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz\") pod \"babf74ed-dd52-4d49-be2c-9e6dd7636442\" (UID: \"babf74ed-dd52-4d49-be2c-9e6dd7636442\") " Jul 24 17:45:44.475568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475527 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:44.475890 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.475718 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.476801 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.476725 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:45:44.478285 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.478235 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:45:44.478597 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.478552 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.479043 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.479015 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.479135 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.479113 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.479895 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.479847 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.479895 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.479858 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:45:44.479895 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.479855 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out" (OuterVolumeSpecName: "config-out") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:45:44.480227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.480200 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume" (OuterVolumeSpecName: "config-volume") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.480515 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.480493 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz" (OuterVolumeSpecName: "kube-api-access-cvjrz") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "kube-api-access-cvjrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:45:44.484135 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.484108 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.490212 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.490187 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config" (OuterVolumeSpecName: "web-config") pod "babf74ed-dd52-4d49-be2c-9e6dd7636442" (UID: "babf74ed-dd52-4d49-be2c-9e6dd7636442"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:45:44.576124 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576084 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/babf74ed-dd52-4d49-be2c-9e6dd7636442-metrics-client-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576124 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576134 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-web-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576144 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-alertmanager-main-db\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576153 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-out\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576162 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-main-tls\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576170 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-cluster-tls-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576178 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvjrz\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-kube-api-access-cvjrz\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576186 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/babf74ed-dd52-4d49-be2c-9e6dd7636442-tls-assets\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576194 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576203 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576212 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:44.576313 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:44.576221 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/babf74ed-dd52-4d49-be2c-9e6dd7636442-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:45:45.136873 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.136838 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" exitCode=0 Jul 24 17:45:45.136873 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.136864 2566 generic.go:356] "Generic (PLEG): container finished" podID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerID="33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" exitCode=0 Jul 24 17:45:45.137076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.136923 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.137076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.136922 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05"} Jul 24 17:45:45.137076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.137010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96"} Jul 24 17:45:45.137076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.137053 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"babf74ed-dd52-4d49-be2c-9e6dd7636442","Type":"ContainerDied","Data":"b1ff8b4d952a32e04b8cff1cf19287f96e3f6a404231b2417e66c6bdf5f5e8fa"} Jul 24 17:45:45.137076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.137071 2566 scope.go:117] "RemoveContainer" containerID="b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" Jul 24 17:45:45.147214 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.147190 2566 scope.go:117] "RemoveContainer" containerID="df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" Jul 24 17:45:45.154295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.154267 2566 scope.go:117] "RemoveContainer" containerID="0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" Jul 24 17:45:45.161140 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.161118 2566 scope.go:117] "RemoveContainer" containerID="33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" Jul 24 17:45:45.167740 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.167714 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:45.168599 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.168582 2566 scope.go:117] "RemoveContainer" containerID="4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" Jul 24 17:45:45.176289 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.176268 2566 scope.go:117] "RemoveContainer" containerID="c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" Jul 24 17:45:45.176440 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.176271 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:45.183367 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.183334 2566 scope.go:117] "RemoveContainer" containerID="b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5" Jul 24 17:45:45.190280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.190256 2566 scope.go:117] "RemoveContainer" containerID="b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" Jul 24 17:45:45.190560 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.190540 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965\": container with ID starting with b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965 not found: ID does not exist" containerID="b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" Jul 24 17:45:45.190647 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.190573 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965"} err="failed to get container status \"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965\": rpc error: code = NotFound desc = could not find container \"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965\": container with ID starting with b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965 not found: ID does not exist" Jul 24 17:45:45.190647 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.190600 2566 scope.go:117] "RemoveContainer" containerID="df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" Jul 24 17:45:45.190853 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.190836 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05\": container with ID starting with df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05 not found: ID does not exist" containerID="df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" Jul 24 17:45:45.190921 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.190862 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05"} err="failed to get container status \"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05\": rpc error: code = NotFound desc = could not find container \"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05\": container with ID starting with df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05 not found: ID does not exist" Jul 24 17:45:45.190921 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.190881 2566 scope.go:117] "RemoveContainer" containerID="0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" Jul 24 17:45:45.191116 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.191096 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a\": container with ID starting with 0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a not found: ID does not exist" containerID="0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" Jul 24 17:45:45.191154 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191123 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a"} err="failed to get container status \"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a\": rpc error: code = NotFound desc = could not find container \"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a\": container with ID starting with 0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a not found: ID does not exist" Jul 24 17:45:45.191154 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191140 2566 scope.go:117] "RemoveContainer" containerID="33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" Jul 24 17:45:45.191385 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.191351 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96\": container with ID starting with 33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96 not found: ID does not exist" containerID="33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" Jul 24 17:45:45.191428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191391 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96"} err="failed to get container status \"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96\": rpc error: code = NotFound desc = could not find container \"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96\": container with ID starting with 33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96 not found: ID does not exist" Jul 24 17:45:45.191428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191406 2566 scope.go:117] "RemoveContainer" containerID="4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" Jul 24 17:45:45.191644 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.191627 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44\": container with ID starting with 4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44 not found: ID does not exist" containerID="4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" Jul 24 17:45:45.191705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191649 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44"} err="failed to get container status \"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44\": rpc error: code = NotFound desc = could not find container \"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44\": container with ID starting with 4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44 not found: ID does not exist" Jul 24 17:45:45.191705 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191668 2566 scope.go:117] "RemoveContainer" containerID="c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" Jul 24 17:45:45.191916 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.191901 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914\": container with ID starting with c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914 not found: ID does not exist" containerID="c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" Jul 24 17:45:45.191957 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191919 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914"} err="failed to get container status \"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914\": rpc error: code = NotFound desc = could not find container \"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914\": container with ID starting with c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914 not found: ID does not exist" Jul 24 17:45:45.191957 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.191933 2566 scope.go:117] "RemoveContainer" containerID="b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5" Jul 24 17:45:45.192113 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:45:45.192099 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5\": container with ID starting with b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5 not found: ID does not exist" containerID="b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5" Jul 24 17:45:45.192152 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192116 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5"} err="failed to get container status \"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5\": rpc error: code = NotFound desc = could not find container \"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5\": container with ID starting with b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5 not found: ID does not exist" Jul 24 17:45:45.192152 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192128 2566 scope.go:117] "RemoveContainer" containerID="b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965" Jul 24 17:45:45.192304 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192287 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965"} err="failed to get container status \"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965\": rpc error: code = NotFound desc = could not find container \"b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965\": container with ID starting with b14db65b479d9a8559d42f2d50120f8b149e13a4947db4baeab0b4a59a009965 not found: ID does not exist" Jul 24 17:45:45.192392 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192308 2566 scope.go:117] "RemoveContainer" containerID="df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05" Jul 24 17:45:45.192548 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192531 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05"} err="failed to get container status \"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05\": rpc error: code = NotFound desc = could not find container \"df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05\": container with ID starting with df0bac2c9b70ad919c03de1b6465a2351bd8e7a7bf1fadbea19daae057a3ab05 not found: ID does not exist" Jul 24 17:45:45.192614 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192549 2566 scope.go:117] "RemoveContainer" containerID="0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a" Jul 24 17:45:45.192787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192771 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a"} err="failed to get container status \"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a\": rpc error: code = NotFound desc = could not find container \"0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a\": container with ID starting with 0a0d237f379c45e3612146de03ee8d3e63720997dd71fa95850c89b8498a8f5a not found: ID does not exist" Jul 24 17:45:45.192831 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192787 2566 scope.go:117] "RemoveContainer" containerID="33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96" Jul 24 17:45:45.192967 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192949 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96"} err="failed to get container status \"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96\": rpc error: code = NotFound desc = could not find container \"33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96\": container with ID starting with 33b2556df621731ae1a1c4f1cd418c9994a506ceafa5d793626ca236cef4aa96 not found: ID does not exist" Jul 24 17:45:45.193030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.192968 2566 scope.go:117] "RemoveContainer" containerID="4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44" Jul 24 17:45:45.193186 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.193169 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44"} err="failed to get container status \"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44\": rpc error: code = NotFound desc = could not find container \"4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44\": container with ID starting with 4ea7dbd83123506f223bede854761e9b92d95d696a2dda18ea4b1e00a0e53d44 not found: ID does not exist" Jul 24 17:45:45.193230 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.193187 2566 scope.go:117] "RemoveContainer" containerID="c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914" Jul 24 17:45:45.193442 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.193423 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914"} err="failed to get container status \"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914\": rpc error: code = NotFound desc = could not find container \"c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914\": container with ID starting with c5328d7437698f04d7e191a7e6b413334cbfb7410c616192af6e4582372ae914 not found: ID does not exist" Jul 24 17:45:45.193515 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.193444 2566 scope.go:117] "RemoveContainer" containerID="b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5" Jul 24 17:45:45.193670 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.193654 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5"} err="failed to get container status \"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5\": rpc error: code = NotFound desc = could not find container \"b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5\": container with ID starting with b3e601bbcd91bfa3b1fe47616a099f8b3ae1cbf05f1e991446f7daec64c995d5 not found: ID does not exist" Jul 24 17:45:45.210808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.210784 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:45.211046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211034 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-metric" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211048 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-metric" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211068 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-web" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211076 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-web" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211087 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="config-reloader" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211095 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="config-reloader" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211108 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="init-config-reloader" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211116 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="init-config-reloader" Jul 24 17:45:45.211129 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211127 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="prom-label-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211136 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="prom-label-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211145 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211151 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211159 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="alertmanager" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211165 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="alertmanager" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211215 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211226 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="config-reloader" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211232 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-web" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211239 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="prom-label-proxy" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211245 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="kube-rbac-proxy-metric" Jul 24 17:45:45.211490 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.211251 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" containerName="alertmanager" Jul 24 17:45:45.216433 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.216415 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.219316 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219290 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 17:45:45.219441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219292 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 17:45:45.219441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219299 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 17:45:45.219441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219336 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 17:45:45.219441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219353 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 17:45:45.219910 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219883 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9z47g\"" Jul 24 17:45:45.220001 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.219938 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 17:45:45.220067 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.220024 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 17:45:45.220272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.220257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 17:45:45.225351 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.225334 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 17:45:45.234503 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.234477 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:45.281708 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281708 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281708 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281729 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-web-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281877 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-config-out\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.281964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6rrn\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-kube-api-access-q6rrn\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.282224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.282224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.281998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.282224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.282025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.282224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.282087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383460 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383460 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383502 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-web-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383816 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.383862 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.383856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-config-out\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.384408 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.384257 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.384408 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.384395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6rrn\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-kube-api-access-q6rrn\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.384604 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.384482 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.387199 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.385672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.387199 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.386589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.388564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d37fdec-e69a-44c9-912e-949d991e220d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.388564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.388564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388415 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-web-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.388564 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388508 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.390548 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388757 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.390548 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.388915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7d37fdec-e69a-44c9-912e-949d991e220d-config-out\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.392838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.392816 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.393094 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.393072 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-config-volume\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.393602 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.393585 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7d37fdec-e69a-44c9-912e-949d991e220d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.398754 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.398728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6rrn\" (UniqueName: \"kubernetes.io/projected/7d37fdec-e69a-44c9-912e-949d991e220d-kube-api-access-q6rrn\") pod \"alertmanager-main-0\" (UID: \"7d37fdec-e69a-44c9-912e-949d991e220d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.525531 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.525480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 17:45:45.664128 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:45.664100 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 17:45:45.665804 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:45:45.665776 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d37fdec_e69a_44c9_912e_949d991e220d.slice/crio-5a9d513b3c09bf8bf481f4afbd669f46065f00906b57866fc228b3cdfd5ac90c WatchSource:0}: Error finding container 5a9d513b3c09bf8bf481f4afbd669f46065f00906b57866fc228b3cdfd5ac90c: Status 404 returned error can't find the container with id 5a9d513b3c09bf8bf481f4afbd669f46065f00906b57866fc228b3cdfd5ac90c Jul 24 17:45:46.142902 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:46.142866 2566 generic.go:356] "Generic (PLEG): container finished" podID="7d37fdec-e69a-44c9-912e-949d991e220d" containerID="642907a9745f9859d3d6970d24d18abced74838ae48c607687f89236763d4eb2" exitCode=0 Jul 24 17:45:46.143068 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:46.142936 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerDied","Data":"642907a9745f9859d3d6970d24d18abced74838ae48c607687f89236763d4eb2"} Jul 24 17:45:46.143068 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:46.142998 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"5a9d513b3c09bf8bf481f4afbd669f46065f00906b57866fc228b3cdfd5ac90c"} Jul 24 17:45:46.249385 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:46.249337 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="babf74ed-dd52-4d49-be2c-9e6dd7636442" path="/var/lib/kubelet/pods/babf74ed-dd52-4d49-be2c-9e6dd7636442/volumes" Jul 24 17:45:47.150559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"25b0569e44915213312838549b21912d724a14e90dce51f313ffe80817c71979"} Jul 24 17:45:47.150990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"1dc2c337eae8cd3a6e068d7e1237e792254323c1e15ca27ec8c27bf8e519c566"} Jul 24 17:45:47.150990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150615 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"656661b166c469950522e2f27daff47fc0f44c772bc4a86f7d5d0b7e84376dd6"} Jul 24 17:45:47.150990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"7e518d08dde8f0ee863c696c00a4b43fd453b30b24e6749926d727e48c04f5cb"} Jul 24 17:45:47.150990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150668 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"68c63b2b812d329abc1eb0953d6b3777d200d93e8e8c2d2f82cb49c120d6f78a"} Jul 24 17:45:47.150990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.150694 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7d37fdec-e69a-44c9-912e-949d991e220d","Type":"ContainerStarted","Data":"16709fbe11aca61e4dbd775ea309ffd5047c05882a7ff2ac722d3fc7edd654d4"} Jul 24 17:45:47.195045 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:47.194977 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.172227998 podStartE2EDuration="2.19495381s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.725812ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:47.193044259 +0000 UTC m=+163.394276668" watchObservedRunningTime="2026-07-24 17:45:47.19495381 +0000 UTC m=+163.396186199" Jul 24 17:45:51.411047 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:51.411010 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:51.411482 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:51.411073 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:51.415707 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:51.415683 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:52.176127 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:52.176096 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:45:52.233477 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:45:52.233445 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:46:17.252826 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.252773 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64c8b575d7-4gz7x" podUID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" containerName="console" containerID="cri-o://816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d" gracePeriod=15 Jul 24 17:46:17.488576 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.488548 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:46:17.545056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.544976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545056 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545022 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dxcr\" (UniqueName: \"kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545062 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545153 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545189 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545255 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle\") pod \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\" (UID: \"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289\") " Jul 24 17:46:17.545554 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:46:17.545554 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545469 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config" (OuterVolumeSpecName: "console-config") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:46:17.545554 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545522 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:46:17.545700 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545564 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-oauth-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.545700 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545582 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.545855 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.545826 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:46:17.547401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.547342 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:46:17.547508 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.547465 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:46:17.547558 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.547498 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr" (OuterVolumeSpecName: "kube-api-access-4dxcr") pod "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" (UID: "5a0ff36a-4d21-4dfc-976d-a58b2f1ec289"). InnerVolumeSpecName "kube-api-access-4dxcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:46:17.646097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.646059 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dxcr\" (UniqueName: \"kubernetes.io/projected/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-kube-api-access-4dxcr\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.646097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.646088 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.646097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.646098 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.646097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.646107 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-console-oauth-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:17.646356 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:17.646138 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289-trusted-ca-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:18.269565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.269527 2566 generic.go:356] "Generic (PLEG): container finished" podID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" containerID="816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d" exitCode=0 Jul 24 17:46:18.269565 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.269568 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c8b575d7-4gz7x" Jul 24 17:46:18.269998 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.269614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c8b575d7-4gz7x" event={"ID":"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289","Type":"ContainerDied","Data":"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d"} Jul 24 17:46:18.269998 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.269662 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c8b575d7-4gz7x" event={"ID":"5a0ff36a-4d21-4dfc-976d-a58b2f1ec289","Type":"ContainerDied","Data":"736b4d1dec726cf3673f3caf891741b31ad7e705c49a3d43343d31456d0244d0"} Jul 24 17:46:18.269998 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.269683 2566 scope.go:117] "RemoveContainer" containerID="816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d" Jul 24 17:46:18.277649 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.277628 2566 scope.go:117] "RemoveContainer" containerID="816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d" Jul 24 17:46:18.277934 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:46:18.277916 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d\": container with ID starting with 816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d not found: ID does not exist" containerID="816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d" Jul 24 17:46:18.277981 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.277943 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d"} err="failed to get container status \"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d\": rpc error: code = NotFound desc = could not find container \"816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d\": container with ID starting with 816c368355e4860bef2de390a74621ec07ab306e40839c4caab19df137421c4d not found: ID does not exist" Jul 24 17:46:18.290318 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.290284 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:46:18.296557 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:18.296535 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64c8b575d7-4gz7x"] Jul 24 17:46:20.247863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:20.247829 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" path="/var/lib/kubelet/pods/5a0ff36a-4d21-4dfc-976d-a58b2f1ec289/volumes" Jul 24 17:46:21.458511 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.458478 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7"] Jul 24 17:46:21.458970 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.458887 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" containerName="console" Jul 24 17:46:21.458970 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.458905 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" containerName="console" Jul 24 17:46:21.458970 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.458964 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a0ff36a-4d21-4dfc-976d-a58b2f1ec289" containerName="console" Jul 24 17:46:21.463720 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.463700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.466528 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.466507 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjdhc\"" Jul 24 17:46:21.466528 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.466526 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 17:46:21.467319 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.467305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 17:46:21.470497 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.470467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6fvt\" (UniqueName: \"kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.470631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.470529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.470631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.470560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.470631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.470620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7"] Jul 24 17:46:21.571432 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.571393 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6fvt\" (UniqueName: \"kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.571432 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.571438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.571632 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.571491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.571825 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.571799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.571929 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.571821 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.580036 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.580011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6fvt\" (UniqueName: \"kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.774189 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.774099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:21.907182 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:21.907160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7"] Jul 24 17:46:21.909492 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:46:21.909469 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4086895_3f23_4bab_ae43_6f6e3e031483.slice/crio-55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76 WatchSource:0}: Error finding container 55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76: Status 404 returned error can't find the container with id 55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76 Jul 24 17:46:22.288460 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:22.288423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" event={"ID":"c4086895-3f23-4bab-ae43-6f6e3e031483","Type":"ContainerStarted","Data":"55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76"} Jul 24 17:46:29.323396 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:29.323340 2566 generic.go:356] "Generic (PLEG): container finished" podID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerID="a67a5a37a7bc327f1ddb45b4b71d1dc74bbca5d253aa1c251ae255dfd9f21fa3" exitCode=0 Jul 24 17:46:29.323396 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:29.323392 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" event={"ID":"c4086895-3f23-4bab-ae43-6f6e3e031483","Type":"ContainerDied","Data":"a67a5a37a7bc327f1ddb45b4b71d1dc74bbca5d253aa1c251ae255dfd9f21fa3"} Jul 24 17:46:32.338145 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:32.338113 2566 generic.go:356] "Generic (PLEG): container finished" podID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerID="7d36a689e5fb71cf3180a44e1cfb79fab6facd9e2059aa5c5c3c0204e0bf3e61" exitCode=0 Jul 24 17:46:32.338537 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:32.338195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" event={"ID":"c4086895-3f23-4bab-ae43-6f6e3e031483","Type":"ContainerDied","Data":"7d36a689e5fb71cf3180a44e1cfb79fab6facd9e2059aa5c5c3c0204e0bf3e61"} Jul 24 17:46:38.366521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:38.366486 2566 generic.go:356] "Generic (PLEG): container finished" podID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerID="638d67a8bc5f1939bfe79c2d66b7d9c3b868a4da4c5e6d02558c90d71bab49b5" exitCode=0 Jul 24 17:46:38.366521 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:38.366517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" event={"ID":"c4086895-3f23-4bab-ae43-6f6e3e031483","Type":"ContainerDied","Data":"638d67a8bc5f1939bfe79c2d66b7d9c3b868a4da4c5e6d02558c90d71bab49b5"} Jul 24 17:46:39.491924 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.491899 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:39.636073 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.635976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle\") pod \"c4086895-3f23-4bab-ae43-6f6e3e031483\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " Jul 24 17:46:39.636073 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.636051 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util\") pod \"c4086895-3f23-4bab-ae43-6f6e3e031483\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " Jul 24 17:46:39.636301 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.636097 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6fvt\" (UniqueName: \"kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt\") pod \"c4086895-3f23-4bab-ae43-6f6e3e031483\" (UID: \"c4086895-3f23-4bab-ae43-6f6e3e031483\") " Jul 24 17:46:39.636700 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.636675 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle" (OuterVolumeSpecName: "bundle") pod "c4086895-3f23-4bab-ae43-6f6e3e031483" (UID: "c4086895-3f23-4bab-ae43-6f6e3e031483"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:46:39.638297 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.638256 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt" (OuterVolumeSpecName: "kube-api-access-n6fvt") pod "c4086895-3f23-4bab-ae43-6f6e3e031483" (UID: "c4086895-3f23-4bab-ae43-6f6e3e031483"). InnerVolumeSpecName "kube-api-access-n6fvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:46:39.641241 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.641220 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util" (OuterVolumeSpecName: "util") pod "c4086895-3f23-4bab-ae43-6f6e3e031483" (UID: "c4086895-3f23-4bab-ae43-6f6e3e031483"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:46:39.737209 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.737176 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6fvt\" (UniqueName: \"kubernetes.io/projected/c4086895-3f23-4bab-ae43-6f6e3e031483-kube-api-access-n6fvt\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:39.737209 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.737204 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:39.737209 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:39.737213 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4086895-3f23-4bab-ae43-6f6e3e031483-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:40.374934 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:40.374906 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" Jul 24 17:46:40.374934 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:40.374922 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j96q7" event={"ID":"c4086895-3f23-4bab-ae43-6f6e3e031483","Type":"ContainerDied","Data":"55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76"} Jul 24 17:46:40.375122 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:40.374963 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55e5a22238d1fdd28b6934f084a1610ee3e0ac3f7ad79623b811893d977b3c76" Jul 24 17:46:45.254673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254638 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k"] Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254933 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="util" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254944 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="util" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254968 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="pull" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254974 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="pull" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254980 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="extract" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.254986 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="extract" Jul 24 17:46:45.255030 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.255027 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4086895-3f23-4bab-ae43-6f6e3e031483" containerName="extract" Jul 24 17:46:45.293196 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.293159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k"] Jul 24 17:46:45.293401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.293289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.296341 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.296317 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 17:46:45.296504 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.296341 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:46:45.296656 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.296632 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sf84g\"" Jul 24 17:46:45.382025 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.381983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqnk\" (UniqueName: \"kubernetes.io/projected/0b3628d9-6b7e-4dc0-896f-c927307c25b1-kube-api-access-xmqnk\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.382173 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.382054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3628d9-6b7e-4dc0-896f-c927307c25b1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.483492 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.483452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3628d9-6b7e-4dc0-896f-c927307c25b1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.483666 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.483520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqnk\" (UniqueName: \"kubernetes.io/projected/0b3628d9-6b7e-4dc0-896f-c927307c25b1-kube-api-access-xmqnk\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.483843 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.483820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3628d9-6b7e-4dc0-896f-c927307c25b1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.492793 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.492768 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqnk\" (UniqueName: \"kubernetes.io/projected/0b3628d9-6b7e-4dc0-896f-c927307c25b1-kube-api-access-xmqnk\") pod \"cert-manager-operator-controller-manager-55c444bf86-z8n7k\" (UID: \"0b3628d9-6b7e-4dc0-896f-c927307c25b1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.603120 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.603094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" Jul 24 17:46:45.781115 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:45.781084 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k"] Jul 24 17:46:45.787588 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:46:45.787548 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b3628d9_6b7e_4dc0_896f_c927307c25b1.slice/crio-798b647265b84f4ffa543495e72470738462b302aea1f72587b2dfcc6fad3cad WatchSource:0}: Error finding container 798b647265b84f4ffa543495e72470738462b302aea1f72587b2dfcc6fad3cad: Status 404 returned error can't find the container with id 798b647265b84f4ffa543495e72470738462b302aea1f72587b2dfcc6fad3cad Jul 24 17:46:46.398173 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:46.398137 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" event={"ID":"0b3628d9-6b7e-4dc0-896f-c927307c25b1","Type":"ContainerStarted","Data":"798b647265b84f4ffa543495e72470738462b302aea1f72587b2dfcc6fad3cad"} Jul 24 17:46:48.407475 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:48.407434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" event={"ID":"0b3628d9-6b7e-4dc0-896f-c927307c25b1","Type":"ContainerStarted","Data":"65d861bbd620a478910b355f32264170a5dae7607f3e572e4030e1d5f01ad236"} Jul 24 17:46:48.435285 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:48.435222 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z8n7k" podStartSLOduration=1.3527477430000001 podStartE2EDuration="3.435201289s" totalImagesPullingTime="2.082453546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:48.432887314 +0000 UTC m=+224.634119695" watchObservedRunningTime="2026-07-24 17:46:48.435201289 +0000 UTC m=+224.636433728" Jul 24 17:46:49.141558 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.141514 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6"] Jul 24 17:46:49.162913 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.162882 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.169276 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.169250 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 17:46:49.169507 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.169251 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 17:46:49.170149 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.170077 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjdhc\"" Jul 24 17:46:49.191623 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.191589 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6"] Jul 24 17:46:49.313791 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.313741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.313983 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.313837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6ktk\" (UniqueName: \"kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.313983 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.313948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.415295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.415203 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.415295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.415256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.415815 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.415312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6ktk\" (UniqueName: \"kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.415815 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.415656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.415815 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.415665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.430025 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.430000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6ktk\" (UniqueName: \"kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.439878 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.439850 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wsn6s"] Jul 24 17:46:49.462815 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.462780 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.464347 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.464319 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wsn6s"] Jul 24 17:46:49.466618 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.466589 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 17:46:49.466745 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.466633 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 17:46:49.472488 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.472467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:49.617116 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.617084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhx9t\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-kube-api-access-mhx9t\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.617265 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.617131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.640626 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.640594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6"] Jul 24 17:46:49.643440 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:46:49.643403 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924f37bc_c658_4256_84c4_7b6f01825f54.slice/crio-4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be WatchSource:0}: Error finding container 4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be: Status 404 returned error can't find the container with id 4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be Jul 24 17:46:49.717819 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.717793 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhx9t\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-kube-api-access-mhx9t\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.717944 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.717836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.729944 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.729915 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.730425 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.730408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhx9t\" (UniqueName: \"kubernetes.io/projected/7eb366a6-bddb-41dd-ab2b-adee6a531db3-kube-api-access-mhx9t\") pod \"cert-manager-webhook-6486ffd997-wsn6s\" (UID: \"7eb366a6-bddb-41dd-ab2b-adee6a531db3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.786872 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.786837 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:49.934843 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:49.934816 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wsn6s"] Jul 24 17:46:49.937233 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:46:49.937208 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eb366a6_bddb_41dd_ab2b_adee6a531db3.slice/crio-7c7bf0036575e67e64c96f9c2d0d5126869219a2c950407652125a4e3cbd8c67 WatchSource:0}: Error finding container 7c7bf0036575e67e64c96f9c2d0d5126869219a2c950407652125a4e3cbd8c67: Status 404 returned error can't find the container with id 7c7bf0036575e67e64c96f9c2d0d5126869219a2c950407652125a4e3cbd8c67 Jul 24 17:46:50.416865 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:50.416810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" event={"ID":"7eb366a6-bddb-41dd-ab2b-adee6a531db3","Type":"ContainerStarted","Data":"7c7bf0036575e67e64c96f9c2d0d5126869219a2c950407652125a4e3cbd8c67"} Jul 24 17:46:50.418187 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:50.418161 2566 generic.go:356] "Generic (PLEG): container finished" podID="924f37bc-c658-4256-84c4-7b6f01825f54" containerID="8c413c67ad3cc28fd9010038f5b719c9c17b91676810916bcf56a5c790b489ad" exitCode=0 Jul 24 17:46:50.418288 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:50.418254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerDied","Data":"8c413c67ad3cc28fd9010038f5b719c9c17b91676810916bcf56a5c790b489ad"} Jul 24 17:46:50.418327 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:50.418292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerStarted","Data":"4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be"} Jul 24 17:46:52.653341 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.653297 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-42s5n"] Jul 24 17:46:52.656794 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.656767 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.659954 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.659926 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-zzcgz\"" Jul 24 17:46:52.664484 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.664457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-42s5n"] Jul 24 17:46:52.744819 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.744722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.744819 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.744770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpph\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-kube-api-access-czpph\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.846119 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.846078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.846119 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.846123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czpph\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-kube-api-access-czpph\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.855280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.855246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.855442 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.855252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpph\" (UniqueName: \"kubernetes.io/projected/776e6f19-433f-4970-8a70-9d9261377dfa-kube-api-access-czpph\") pod \"cert-manager-cainjector-559cdd8c87-42s5n\" (UID: \"776e6f19-433f-4970-8a70-9d9261377dfa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:52.971431 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:52.971392 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" Jul 24 17:46:53.436831 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:53.436796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerStarted","Data":"bbd765c2f5ab0903291cd38684ad46b6db518252d822142b3a0838874b1c480a"} Jul 24 17:46:53.499727 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:53.499691 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-42s5n"] Jul 24 17:46:53.520401 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:46:53.520352 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod776e6f19_433f_4970_8a70_9d9261377dfa.slice/crio-5ad6e467a2ec952eb17c55452eb1ae5df437e807fb3d0ca9be5d5af475988af1 WatchSource:0}: Error finding container 5ad6e467a2ec952eb17c55452eb1ae5df437e807fb3d0ca9be5d5af475988af1: Status 404 returned error can't find the container with id 5ad6e467a2ec952eb17c55452eb1ae5df437e807fb3d0ca9be5d5af475988af1 Jul 24 17:46:54.442541 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.442504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" event={"ID":"7eb366a6-bddb-41dd-ab2b-adee6a531db3","Type":"ContainerStarted","Data":"f4731a8d2050f40e79bac787bbab1323c428df5a623bb843bd22068b443da011"} Jul 24 17:46:54.443002 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.442619 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:46:54.444296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.444272 2566 generic.go:356] "Generic (PLEG): container finished" podID="924f37bc-c658-4256-84c4-7b6f01825f54" containerID="bbd765c2f5ab0903291cd38684ad46b6db518252d822142b3a0838874b1c480a" exitCode=0 Jul 24 17:46:54.444476 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.444407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerDied","Data":"bbd765c2f5ab0903291cd38684ad46b6db518252d822142b3a0838874b1c480a"} Jul 24 17:46:54.445942 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.445919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" event={"ID":"776e6f19-433f-4970-8a70-9d9261377dfa","Type":"ContainerStarted","Data":"07bbb558900d59dfad6be1d1f82dd1e99d4230f7ffd8cfea93c6513067887659"} Jul 24 17:46:54.446053 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.445948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" event={"ID":"776e6f19-433f-4970-8a70-9d9261377dfa","Type":"ContainerStarted","Data":"5ad6e467a2ec952eb17c55452eb1ae5df437e807fb3d0ca9be5d5af475988af1"} Jul 24 17:46:54.458987 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.458938 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" podStartSLOduration=2.000964547 podStartE2EDuration="5.458918784s" totalImagesPullingTime="3.457954237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:54.458244376 +0000 UTC m=+230.659476781" watchObservedRunningTime="2026-07-24 17:46:54.458918784 +0000 UTC m=+230.660151176" Jul 24 17:46:54.490899 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:54.490850 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-42s5n" podStartSLOduration=2.490831984 podStartE2EDuration="2.490831984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:54.48995468 +0000 UTC m=+230.691187069" watchObservedRunningTime="2026-07-24 17:46:54.490831984 +0000 UTC m=+230.692064373" Jul 24 17:46:55.457578 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:55.457538 2566 generic.go:356] "Generic (PLEG): container finished" podID="924f37bc-c658-4256-84c4-7b6f01825f54" containerID="ac701e0102e731069a6cb3b92c804730abbfaf2875d2c0e3f1ac711d7ea71618" exitCode=0 Jul 24 17:46:55.458052 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:55.457633 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerDied","Data":"ac701e0102e731069a6cb3b92c804730abbfaf2875d2c0e3f1ac711d7ea71618"} Jul 24 17:46:56.587013 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.586973 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:56.680522 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.680476 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util\") pod \"924f37bc-c658-4256-84c4-7b6f01825f54\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " Jul 24 17:46:56.680719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.680542 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle\") pod \"924f37bc-c658-4256-84c4-7b6f01825f54\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " Jul 24 17:46:56.680719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.680574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6ktk\" (UniqueName: \"kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk\") pod \"924f37bc-c658-4256-84c4-7b6f01825f54\" (UID: \"924f37bc-c658-4256-84c4-7b6f01825f54\") " Jul 24 17:46:56.681005 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.680969 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle" (OuterVolumeSpecName: "bundle") pod "924f37bc-c658-4256-84c4-7b6f01825f54" (UID: "924f37bc-c658-4256-84c4-7b6f01825f54"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:46:56.682759 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.682738 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk" (OuterVolumeSpecName: "kube-api-access-z6ktk") pod "924f37bc-c658-4256-84c4-7b6f01825f54" (UID: "924f37bc-c658-4256-84c4-7b6f01825f54"). InnerVolumeSpecName "kube-api-access-z6ktk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:46:56.685495 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.685455 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util" (OuterVolumeSpecName: "util") pod "924f37bc-c658-4256-84c4-7b6f01825f54" (UID: "924f37bc-c658-4256-84c4-7b6f01825f54"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:46:56.781170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.781076 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6ktk\" (UniqueName: \"kubernetes.io/projected/924f37bc-c658-4256-84c4-7b6f01825f54-kube-api-access-z6ktk\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:56.781170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.781112 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:56.781170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:56.781122 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/924f37bc-c658-4256-84c4-7b6f01825f54-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:46:57.468601 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:57.468570 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" Jul 24 17:46:57.468801 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:57.468572 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmn8b6" event={"ID":"924f37bc-c658-4256-84c4-7b6f01825f54","Type":"ContainerDied","Data":"4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be"} Jul 24 17:46:57.468801 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:57.468688 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bb513b4e17c536682a3e8987b4117d5b15cb4cd2384975c436e94fe587b92be" Jul 24 17:46:59.826912 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.826879 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vs8tn"] Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827168 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="extract" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827179 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="extract" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827188 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="util" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827193 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="util" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827213 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="pull" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827219 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="pull" Jul 24 17:46:59.827322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.827270 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="924f37bc-c658-4256-84c4-7b6f01825f54" containerName="extract" Jul 24 17:46:59.829121 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.829102 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:46:59.831598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.831575 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-vq869\"" Jul 24 17:46:59.839017 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:46:59.838973 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vs8tn"] Jul 24 17:47:00.007170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.007125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrr9\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-kube-api-access-phrr9\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.007349 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.007238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-bound-sa-token\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.108403 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.108346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrr9\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-kube-api-access-phrr9\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.108592 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.108447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-bound-sa-token\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.117266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.117230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrr9\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-kube-api-access-phrr9\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.117266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.117254 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f07361d3-db35-4810-99ba-527d8ac43e3c-bound-sa-token\") pod \"cert-manager-55b4d954d5-vs8tn\" (UID: \"f07361d3-db35-4810-99ba-527d8ac43e3c\") " pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.144293 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.144254 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vs8tn" Jul 24 17:47:00.277583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.277554 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vs8tn"] Jul 24 17:47:00.279784 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:00.279751 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf07361d3_db35_4810_99ba_527d8ac43e3c.slice/crio-446c04fd619d0c699d852f3466b9ad985a5b5bcefa8c2dcc9e7d6e14828c20dc WatchSource:0}: Error finding container 446c04fd619d0c699d852f3466b9ad985a5b5bcefa8c2dcc9e7d6e14828c20dc: Status 404 returned error can't find the container with id 446c04fd619d0c699d852f3466b9ad985a5b5bcefa8c2dcc9e7d6e14828c20dc Jul 24 17:47:00.460263 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.459956 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wsn6s" Jul 24 17:47:00.486006 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.485969 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vs8tn" event={"ID":"f07361d3-db35-4810-99ba-527d8ac43e3c","Type":"ContainerStarted","Data":"aece93905eb4ad515815e955bcfcbf9134c3d463fb47762a9e61c80e0ff17fdd"} Jul 24 17:47:00.486006 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.486012 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vs8tn" event={"ID":"f07361d3-db35-4810-99ba-527d8ac43e3c","Type":"ContainerStarted","Data":"446c04fd619d0c699d852f3466b9ad985a5b5bcefa8c2dcc9e7d6e14828c20dc"} Jul 24 17:47:00.509157 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:00.509110 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vs8tn" podStartSLOduration=1.5090942059999999 podStartE2EDuration="1.509094206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:00.508715123 +0000 UTC m=+236.709947532" watchObservedRunningTime="2026-07-24 17:47:00.509094206 +0000 UTC m=+236.710326594" Jul 24 17:47:02.892706 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.892664 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2"] Jul 24 17:47:02.895074 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.895051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:02.897816 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.897793 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 17:47:02.899051 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.899031 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f8lkl\"" Jul 24 17:47:02.899157 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.899093 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 17:47:02.903545 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:02.903519 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2"] Jul 24 17:47:03.032529 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.032495 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pwql\" (UniqueName: \"kubernetes.io/projected/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-kube-api-access-9pwql\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.032692 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.032580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.133172 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.133117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pwql\" (UniqueName: \"kubernetes.io/projected/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-kube-api-access-9pwql\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.133330 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.133216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.133593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.133576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.141986 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.141956 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pwql\" (UniqueName: \"kubernetes.io/projected/e7770bc7-3eff-4bf8-85eb-eee11a50cc22-kube-api-access-9pwql\") pod \"openshift-lws-operator-fd8ccff4c-cq9f2\" (UID: \"e7770bc7-3eff-4bf8-85eb-eee11a50cc22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.204855 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.204765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" Jul 24 17:47:03.338923 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.338847 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2"] Jul 24 17:47:03.341341 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:03.341316 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7770bc7_3eff_4bf8_85eb_eee11a50cc22.slice/crio-8a735b9ef624f05aaf995e0a2690ec597364c030319fea5852dfe81f36bb6297 WatchSource:0}: Error finding container 8a735b9ef624f05aaf995e0a2690ec597364c030319fea5852dfe81f36bb6297: Status 404 returned error can't find the container with id 8a735b9ef624f05aaf995e0a2690ec597364c030319fea5852dfe81f36bb6297 Jul 24 17:47:03.500050 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:03.499960 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" event={"ID":"e7770bc7-3eff-4bf8-85eb-eee11a50cc22","Type":"ContainerStarted","Data":"8a735b9ef624f05aaf995e0a2690ec597364c030319fea5852dfe81f36bb6297"} Jul 24 17:47:05.510399 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:05.510335 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" event={"ID":"e7770bc7-3eff-4bf8-85eb-eee11a50cc22","Type":"ContainerStarted","Data":"98833b1d8bcfefb4c3249d895a646ff948c63256ae5ca3230a457adf55329416"} Jul 24 17:47:05.527607 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:05.527556 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cq9f2" podStartSLOduration=1.8445433260000001 podStartE2EDuration="3.527540536s" totalImagesPullingTime="1.68299721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:05.526742731 +0000 UTC m=+241.727975158" watchObservedRunningTime="2026-07-24 17:47:05.527540536 +0000 UTC m=+241.728772923" Jul 24 17:47:08.056324 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.056287 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52"] Jul 24 17:47:08.058958 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.058939 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.061933 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.061906 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 17:47:08.062062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.061906 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 17:47:08.063035 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.063016 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjdhc\"" Jul 24 17:47:08.068175 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.068149 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52"] Jul 24 17:47:08.068528 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.068500 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tc5p\" (UniqueName: \"kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.068613 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.068574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.068673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.068610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.169272 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.169230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tc5p\" (UniqueName: \"kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.169501 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.169286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.169501 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.169313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.169771 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.169748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.169846 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.169812 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.177916 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.177885 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tc5p\" (UniqueName: \"kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.370183 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.370142 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:08.507787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.507758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52"] Jul 24 17:47:08.510449 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:08.510410 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebbdbb13_761e_4dbb_9c3f_bae1e1b495ed.slice/crio-6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a WatchSource:0}: Error finding container 6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a: Status 404 returned error can't find the container with id 6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a Jul 24 17:47:08.527424 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:08.527396 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" event={"ID":"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed","Type":"ContainerStarted","Data":"6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a"} Jul 24 17:47:09.532949 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:09.532860 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerID="1483c73b2bb57336640b91c8f16283a7aef769faf1edc57c6913f1eb17615a03" exitCode=0 Jul 24 17:47:09.532949 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:09.532933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" event={"ID":"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed","Type":"ContainerDied","Data":"1483c73b2bb57336640b91c8f16283a7aef769faf1edc57c6913f1eb17615a03"} Jul 24 17:47:10.539531 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:10.539498 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerID="74c4b4d9f56b5bf48e85d368bf18457ee7c0127fc84887744a97ab3c1b004c61" exitCode=0 Jul 24 17:47:10.539923 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:10.539591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" event={"ID":"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed","Type":"ContainerDied","Data":"74c4b4d9f56b5bf48e85d368bf18457ee7c0127fc84887744a97ab3c1b004c61"} Jul 24 17:47:11.545497 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:11.545457 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerID="a0430820bfc637ad3ffbbb70b66148e275a1fd45b0dbd9a7a91e56fc204b44e4" exitCode=0 Jul 24 17:47:11.545896 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:11.545541 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" event={"ID":"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed","Type":"ContainerDied","Data":"a0430820bfc637ad3ffbbb70b66148e275a1fd45b0dbd9a7a91e56fc204b44e4"} Jul 24 17:47:12.678017 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.677991 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:12.700791 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.700765 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle\") pod \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " Jul 24 17:47:12.700932 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.700811 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util\") pod \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " Jul 24 17:47:12.700932 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.700873 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tc5p\" (UniqueName: \"kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p\") pod \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\" (UID: \"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed\") " Jul 24 17:47:12.701603 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.701574 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle" (OuterVolumeSpecName: "bundle") pod "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" (UID: "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:47:12.702960 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.702929 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p" (OuterVolumeSpecName: "kube-api-access-4tc5p") pod "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" (UID: "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed"). InnerVolumeSpecName "kube-api-access-4tc5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:47:12.706665 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.706624 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util" (OuterVolumeSpecName: "util") pod "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" (UID: "ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:47:12.801455 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.801349 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tc5p\" (UniqueName: \"kubernetes.io/projected/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-kube-api-access-4tc5p\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:47:12.801455 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.801407 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:47:12.801455 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:12.801417 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:47:13.554948 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:13.554921 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" Jul 24 17:47:13.554948 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:13.554927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afclb52" event={"ID":"ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed","Type":"ContainerDied","Data":"6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a"} Jul 24 17:47:13.555193 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:13.554980 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f4481d33795ae3f31c834bc29247e37b407bcdd96b149e850f1025face2a54a" Jul 24 17:47:24.737164 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737122 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p"] Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737482 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="pull" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737497 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="pull" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737511 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="util" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737521 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="util" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737541 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="extract" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737551 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="extract" Jul 24 17:47:24.737673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.737634 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebbdbb13-761e-4dbb-9c3f-bae1e1b495ed" containerName="extract" Jul 24 17:47:24.742324 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.742305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.749889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.749722 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 17:47:24.749889 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.749722 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 17:47:24.750082 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.750022 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vnvl4\"" Jul 24 17:47:24.750305 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.750128 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 17:47:24.750305 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.750240 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 17:47:24.776980 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.776945 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p"] Jul 24 17:47:24.793258 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.793224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.793432 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.793272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-tmp\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.793432 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.793350 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96qvz\" (UniqueName: \"kubernetes.io/projected/7c020c2e-014d-4a3c-be20-5681527c383c-kube-api-access-96qvz\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.793432 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.793396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.793583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.793468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-manifests\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894310 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894310 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-tmp\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894582 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96qvz\" (UniqueName: \"kubernetes.io/projected/7c020c2e-014d-4a3c-be20-5681527c383c-kube-api-access-96qvz\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894582 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894582 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-manifests\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.894883 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.894863 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-manifests\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.896581 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.896555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c020c2e-014d-4a3c-be20-5681527c383c-tmp\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.896829 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.896811 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.896895 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.896818 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c020c2e-014d-4a3c-be20-5681527c383c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:24.906240 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:24.906212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96qvz\" (UniqueName: \"kubernetes.io/projected/7c020c2e-014d-4a3c-be20-5681527c383c-kube-api-access-96qvz\") pod \"opendatahub-operator-controller-manager-dbf4bd545-l2s9p\" (UID: \"7c020c2e-014d-4a3c-be20-5681527c383c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:25.054675 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:25.054575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:25.205886 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:25.205861 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p"] Jul 24 17:47:25.208179 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:25.208153 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c020c2e_014d_4a3c_be20_5681527c383c.slice/crio-5e5cacf35d307bebde22c1f3b7c9ea28db5debd35ddd6d1465fe9c1b7d1f530e WatchSource:0}: Error finding container 5e5cacf35d307bebde22c1f3b7c9ea28db5debd35ddd6d1465fe9c1b7d1f530e: Status 404 returned error can't find the container with id 5e5cacf35d307bebde22c1f3b7c9ea28db5debd35ddd6d1465fe9c1b7d1f530e Jul 24 17:47:25.609037 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:25.609002 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" event={"ID":"7c020c2e-014d-4a3c-be20-5681527c383c","Type":"ContainerStarted","Data":"5e5cacf35d307bebde22c1f3b7c9ea28db5debd35ddd6d1465fe9c1b7d1f530e"} Jul 24 17:47:28.625929 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:28.625892 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" event={"ID":"7c020c2e-014d-4a3c-be20-5681527c383c","Type":"ContainerStarted","Data":"7272f081e4ca3713b097d56bc7ef13e11d43ccdf6d638d7a2c0ad7291485c027"} Jul 24 17:47:29.634835 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:29.634803 2566 generic.go:356] "Generic (PLEG): container finished" podID="7c020c2e-014d-4a3c-be20-5681527c383c" containerID="7272f081e4ca3713b097d56bc7ef13e11d43ccdf6d638d7a2c0ad7291485c027" exitCode=0 Jul 24 17:47:29.635192 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:29.634890 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" event={"ID":"7c020c2e-014d-4a3c-be20-5681527c383c","Type":"ContainerDied","Data":"7272f081e4ca3713b097d56bc7ef13e11d43ccdf6d638d7a2c0ad7291485c027"} Jul 24 17:47:30.641611 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:30.641562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" event={"ID":"7c020c2e-014d-4a3c-be20-5681527c383c","Type":"ContainerStarted","Data":"a3470e382729e0d8de3784f1fefd18bdacf0a2f3a3e030aa3ea906904b42faee"} Jul 24 17:47:30.641987 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:30.641667 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:30.701739 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:30.701674 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" podStartSLOduration=2.046264531 podStartE2EDuration="6.701656915s" totalImagesPullingTime="3.489049354s" totalInitContainerRuntime="1.16634303s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:30.698675008 +0000 UTC m=+266.899907416" watchObservedRunningTime="2026-07-24 17:47:30.701656915 +0000 UTC m=+266.902889440" Jul 24 17:47:41.648632 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:41.648587 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf4bd545-l2s9p" Jul 24 17:47:44.627633 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.627594 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p"] Jul 24 17:47:44.634087 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.634058 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.636967 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.636936 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 17:47:44.637120 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.637012 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xcmzs\"" Jul 24 17:47:44.638055 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.638035 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 17:47:44.638204 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.638114 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 17:47:44.638269 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.638138 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 17:47:44.649838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.649811 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p"] Jul 24 17:47:44.768780 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.768736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tls-certs\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.768955 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.768813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr527\" (UniqueName: \"kubernetes.io/projected/be8c0bad-e930-40b1-8cde-cf68fd838e0a-kube-api-access-fr527\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.768955 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.768880 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tmp\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.869805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.869762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tmp\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.870012 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.869839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tls-certs\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.870012 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.869883 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr527\" (UniqueName: \"kubernetes.io/projected/be8c0bad-e930-40b1-8cde-cf68fd838e0a-kube-api-access-fr527\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.872120 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.872089 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tmp\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.872440 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.872414 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be8c0bad-e930-40b1-8cde-cf68fd838e0a-tls-certs\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.880436 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.880352 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr527\" (UniqueName: \"kubernetes.io/projected/be8c0bad-e930-40b1-8cde-cf68fd838e0a-kube-api-access-fr527\") pod \"kube-auth-proxy-56c646dc89-jpn2p\" (UID: \"be8c0bad-e930-40b1-8cde-cf68fd838e0a\") " pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:44.946851 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:44.946819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" Jul 24 17:47:45.100270 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.100246 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p"] Jul 24 17:47:45.101899 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:45.101870 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8c0bad_e930_40b1_8cde_cf68fd838e0a.slice/crio-ef7fd47980fc14c266de8b590a390f464fda32c7a647e1117a01c3e5fe081fd7 WatchSource:0}: Error finding container ef7fd47980fc14c266de8b590a390f464fda32c7a647e1117a01c3e5fe081fd7: Status 404 returned error can't find the container with id ef7fd47980fc14c266de8b590a390f464fda32c7a647e1117a01c3e5fe081fd7 Jul 24 17:47:45.716158 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.716114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" event={"ID":"be8c0bad-e930-40b1-8cde-cf68fd838e0a","Type":"ContainerStarted","Data":"ef7fd47980fc14c266de8b590a390f464fda32c7a647e1117a01c3e5fe081fd7"} Jul 24 17:47:45.860788 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.860749 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l4szd"] Jul 24 17:47:45.863376 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.863335 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:45.865876 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.865851 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 17:47:45.866437 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.866416 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-572wg\"" Jul 24 17:47:45.883038 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.876584 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l4szd"] Jul 24 17:47:45.980559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.980478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:45.980559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:45.980535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xql5z\" (UniqueName: \"kubernetes.io/projected/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-kube-api-access-xql5z\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.081444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.081398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.081630 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.081456 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xql5z\" (UniqueName: \"kubernetes.io/projected/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-kube-api-access-xql5z\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.081630 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:47:46.081605 2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 24 17:47:46.081716 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:47:46.081702 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs podName:800f4d5e-f2ff-4f6c-b6cd-f449dd720137 nodeName:}" failed. No retries permitted until 2026-07-24 17:47:46.581679166 +0000 UTC m=+282.782911550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs") pod "model-serving-api-54d89bd5fc-l4szd" (UID: "800f4d5e-f2ff-4f6c-b6cd-f449dd720137") : secret "model-serving-api-tls" not found Jul 24 17:47:46.093425 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.093391 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xql5z\" (UniqueName: \"kubernetes.io/projected/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-kube-api-access-xql5z\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.585154 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.585110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.588044 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.588009 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800f4d5e-f2ff-4f6c-b6cd-f449dd720137-tls-certs\") pod \"model-serving-api-54d89bd5fc-l4szd\" (UID: \"800f4d5e-f2ff-4f6c-b6cd-f449dd720137\") " pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:46.777670 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:46.777633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:47.647972 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:47.647883 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l4szd"] Jul 24 17:47:47.650486 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:47.650454 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800f4d5e_f2ff_4f6c_b6cd_f449dd720137.slice/crio-316ebe49cfc963c97f7cb975ea8970465a7929721f9bea97730f84329ab546aa WatchSource:0}: Error finding container 316ebe49cfc963c97f7cb975ea8970465a7929721f9bea97730f84329ab546aa: Status 404 returned error can't find the container with id 316ebe49cfc963c97f7cb975ea8970465a7929721f9bea97730f84329ab546aa Jul 24 17:47:47.726691 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:47.726648 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" event={"ID":"800f4d5e-f2ff-4f6c-b6cd-f449dd720137","Type":"ContainerStarted","Data":"316ebe49cfc963c97f7cb975ea8970465a7929721f9bea97730f84329ab546aa"} Jul 24 17:47:47.728337 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:47.728311 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" event={"ID":"be8c0bad-e930-40b1-8cde-cf68fd838e0a","Type":"ContainerStarted","Data":"9cf7b074da740791c72d3503351a7ed93e6d1887c9939b6e5cc6bc49d18ad114"} Jul 24 17:47:47.746694 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:47.746579 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56c646dc89-jpn2p" podStartSLOduration=1.3292206 podStartE2EDuration="3.746558413s" totalImagesPullingTime="2.417337813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:47.746225431 +0000 UTC m=+283.947457818" watchObservedRunningTime="2026-07-24 17:47:47.746558413 +0000 UTC m=+283.947790797" Jul 24 17:47:49.740792 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:49.740752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" event={"ID":"800f4d5e-f2ff-4f6c-b6cd-f449dd720137","Type":"ContainerStarted","Data":"090144b5c002989f5d94e5024e317eddd2658612dbf0341d527653df388f81ac"} Jul 24 17:47:49.741260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:49.740859 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:47:49.758145 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:49.758093 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" podStartSLOduration=3.422191271 podStartE2EDuration="4.758076657s" totalImagesPullingTime="1.335885386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:49.756661615 +0000 UTC m=+285.957894001" watchObservedRunningTime="2026-07-24 17:47:49.758076657 +0000 UTC m=+285.959309047" Jul 24 17:47:51.196063 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.196025 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:47:51.198656 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.198635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.201051 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.201023 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 17:47:51.201350 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.201328 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7mhq7\"" Jul 24 17:47:51.210854 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.210779 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:47:51.331605 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331611 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcvp9\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331781 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331999 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331999 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331999 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.331999 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.331946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433222 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433308 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433648 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcvp9\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.433984 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.433909 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.434153 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.434114 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.434301 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.434276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.434404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.434326 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.434404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.434388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.435772 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.435750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.436554 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.436535 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.450655 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.450581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.450892 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.450870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcvp9\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.510580 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.510540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:51.650340 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.650309 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:47:51.653183 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:51.653156 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaccf825_c3cd_40b2_85fe_1ef31317a7cd.slice/crio-04a6a42457375eec1dc9f18b140030c9b94d9433fd800b9c4c8b09af88ac7596 WatchSource:0}: Error finding container 04a6a42457375eec1dc9f18b140030c9b94d9433fd800b9c4c8b09af88ac7596: Status 404 returned error can't find the container with id 04a6a42457375eec1dc9f18b140030c9b94d9433fd800b9c4c8b09af88ac7596 Jul 24 17:47:51.750969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:51.750883 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" event={"ID":"baccf825-c3cd-40b2-85fe-1ef31317a7cd","Type":"ContainerStarted","Data":"04a6a42457375eec1dc9f18b140030c9b94d9433fd800b9c4c8b09af88ac7596"} Jul 24 17:47:54.048230 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.048194 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf"] Jul 24 17:47:54.053582 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.053559 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.057502 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.057343 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dbqjz\"" Jul 24 17:47:54.057502 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.057466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 17:47:54.057700 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.057577 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 17:47:54.057811 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.057796 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 17:47:54.066440 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.066410 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf"] Jul 24 17:47:54.158673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.158638 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4xf\" (UniqueName: \"kubernetes.io/projected/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-kube-api-access-fk4xf\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.158849 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.158750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-metrics-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.158849 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.158785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-manager-config\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.158956 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.158900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.260073 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.260044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-metrics-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.260260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.260086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-manager-config\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.260260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.260131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.260260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.260167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4xf\" (UniqueName: \"kubernetes.io/projected/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-kube-api-access-fk4xf\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.260831 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.260810 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-manager-config\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.262691 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.262665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.262787 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.262665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-metrics-cert\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.275721 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.275692 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4xf\" (UniqueName: \"kubernetes.io/projected/2602affc-9630-46ca-a0d1-cfef2fbf4b7f-kube-api-access-fk4xf\") pod \"lws-controller-manager-9f897544f-mk7mf\" (UID: \"2602affc-9630-46ca-a0d1-cfef2fbf4b7f\") " pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.302727 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.302677 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:47:54.302805 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.302765 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:47:54.302841 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.302803 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 17:47:54.367386 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.367344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:54.549247 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.549208 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf"] Jul 24 17:47:54.549395 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:47:54.549378 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2602affc_9630_46ca_a0d1_cfef2fbf4b7f.slice/crio-0ed52b56b8ea1e913767007bd75d9222a8a047ed9af1b7daaa34ab9543d239c2 WatchSource:0}: Error finding container 0ed52b56b8ea1e913767007bd75d9222a8a047ed9af1b7daaa34ab9543d239c2: Status 404 returned error can't find the container with id 0ed52b56b8ea1e913767007bd75d9222a8a047ed9af1b7daaa34ab9543d239c2 Jul 24 17:47:54.768080 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.768036 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" event={"ID":"baccf825-c3cd-40b2-85fe-1ef31317a7cd","Type":"ContainerStarted","Data":"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36"} Jul 24 17:47:54.769278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.769250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" event={"ID":"2602affc-9630-46ca-a0d1-cfef2fbf4b7f","Type":"ContainerStarted","Data":"0ed52b56b8ea1e913767007bd75d9222a8a047ed9af1b7daaa34ab9543d239c2"} Jul 24 17:47:54.789510 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:54.789439 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" podStartSLOduration=1.142290989 podStartE2EDuration="3.789417703s" totalImagesPullingTime="2.647126714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:54.78847906 +0000 UTC m=+290.989711450" watchObservedRunningTime="2026-07-24 17:47:54.789417703 +0000 UTC m=+290.990650093" Jul 24 17:47:55.511318 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:55.511281 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:47:55.512907 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:55.512868 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 17:47:55.513046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:55.512938 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 17:47:56.162029 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.161990 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:47:56.511446 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.511413 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 17:47:56.511823 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.511471 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 17:47:56.784178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.784075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" event={"ID":"2602affc-9630-46ca-a0d1-cfef2fbf4b7f","Type":"ContainerStarted","Data":"a491b282146c115c77e8ccf6e59ab4c2510b7023fc5d8c9713e0987c9e154af1"} Jul 24 17:47:56.784376 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.784227 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:47:56.824917 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:56.824861 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" podStartSLOduration=1.366852744 podStartE2EDuration="2.824841807s" totalImagesPullingTime="1.457989063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:47:56.824018516 +0000 UTC m=+293.025250905" watchObservedRunningTime="2026-07-24 17:47:56.824841807 +0000 UTC m=+293.026074195" Jul 24 17:47:57.511817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:57.511778 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 24 17:47:57.512199 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:57.511863 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 24 17:47:57.788559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:47:57.788450 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" containerID="cri-o://7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36" gracePeriod=30 Jul 24 17:48:00.749113 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:00.749078 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-l4szd" Jul 24 17:48:03.026403 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.026377 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:48:03.037266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037244 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037377 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037291 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037377 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037313 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037377 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037332 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037377 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037349 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037388 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037434 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037477 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037499 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcvp9\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9\") pod \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\" (UID: \"baccf825-c3cd-40b2-85fe-1ef31317a7cd\") " Jul 24 17:48:03.037562 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037528 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:03.037808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037737 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.037808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037701 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:03.037808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037756 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:03.037808 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037788 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:48:03.037962 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.037924 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:03.039536 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.039506 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 17:48:03.039685 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.039666 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:03.039756 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.039682 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:03.039756 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.039739 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9" (OuterVolumeSpecName: "kube-api-access-zcvp9") pod "baccf825-c3cd-40b2-85fe-1ef31317a7cd" (UID: "baccf825-c3cd-40b2-85fe-1ef31317a7cd"). InnerVolumeSpecName "kube-api-access-zcvp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:03.138295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138254 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138287 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138295 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138299 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138311 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138324 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138334 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138344 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcvp9\" (UniqueName: \"kubernetes.io/projected/baccf825-c3cd-40b2-85fe-1ef31317a7cd-kube-api-access-zcvp9\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.138583 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.138355 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baccf825-c3cd-40b2-85fe-1ef31317a7cd-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:03.820704 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.820670 2566 generic.go:356] "Generic (PLEG): container finished" podID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerID="7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36" exitCode=0 Jul 24 17:48:03.820877 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.820716 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" Jul 24 17:48:03.820877 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.820738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" event={"ID":"baccf825-c3cd-40b2-85fe-1ef31317a7cd","Type":"ContainerDied","Data":"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36"} Jul 24 17:48:03.820877 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.820796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m" event={"ID":"baccf825-c3cd-40b2-85fe-1ef31317a7cd","Type":"ContainerDied","Data":"04a6a42457375eec1dc9f18b140030c9b94d9433fd800b9c4c8b09af88ac7596"} Jul 24 17:48:03.820877 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.820820 2566 scope.go:117] "RemoveContainer" containerID="7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36" Jul 24 17:48:03.829989 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.829961 2566 scope.go:117] "RemoveContainer" containerID="7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36" Jul 24 17:48:03.830254 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:48:03.830233 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36\": container with ID starting with 7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36 not found: ID does not exist" containerID="7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36" Jul 24 17:48:03.830297 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.830262 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36"} err="failed to get container status \"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36\": rpc error: code = NotFound desc = could not find container \"7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36\": container with ID starting with 7cab92b3d94b6d30a3429d0685fac0942f91664af44fbada4893aea39f344e36 not found: ID does not exist" Jul 24 17:48:03.851836 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.851807 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:48:03.855942 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:03.855919 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcwgw5m"] Jul 24 17:48:04.164715 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:04.164687 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 17:48:04.249974 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:04.249818 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" path="/var/lib/kubelet/pods/baccf825-c3cd-40b2-85fe-1ef31317a7cd/volumes" Jul 24 17:48:07.790772 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:07.790741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9f897544f-mk7mf" Jul 24 17:48:49.534659 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.534622 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x"] Jul 24 17:48:49.535084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.535030 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" Jul 24 17:48:49.535084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.535046 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" Jul 24 17:48:49.535154 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.535117 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="baccf825-c3cd-40b2-85fe-1ef31317a7cd" containerName="istio-proxy" Jul 24 17:48:49.540845 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.540823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.544238 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.544215 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 17:48:49.544396 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.544261 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 17:48:49.545279 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.545262 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tjdhc\"" Jul 24 17:48:49.550662 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.550641 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x"] Jul 24 17:48:49.594496 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.594461 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp"] Jul 24 17:48:49.597046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.597022 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.609456 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.609430 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp"] Jul 24 17:48:49.646412 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bkvj\" (UniqueName: \"kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.646593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.646593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646465 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.646593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.646593 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89mkj\" (UniqueName: \"kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.646754 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.646620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.679444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.679407 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl"] Jul 24 17:48:49.682061 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.682044 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.692964 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.692931 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl"] Jul 24 17:48:49.747657 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q89tk\" (UniqueName: \"kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89mkj\" (UniqueName: \"kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.747860 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.748174 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.747963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.748174 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.748014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bkvj\" (UniqueName: \"kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.748174 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.748083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.748174 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.748125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.748298 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.748190 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.748298 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.748289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.757401 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.757347 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89mkj\" (UniqueName: \"kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.760009 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.759975 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bkvj\" (UniqueName: \"kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.790303 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.790219 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml"] Jul 24 17:48:49.793076 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.793048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.802826 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.802792 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml"] Jul 24 17:48:49.848959 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.848921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q89tk\" (UniqueName: \"kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.848959 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.848963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.849224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.849224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.849224 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgwpm\" (UniqueName: \"kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.849394 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.849509 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.849567 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.849550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.850877 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.850864 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:49.858469 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.858445 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q89tk\" (UniqueName: \"kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:49.905780 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.905745 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:49.950390 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.950312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.950390 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.950353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgwpm\" (UniqueName: \"kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.950666 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.950503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.950949 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.950921 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.950949 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.950946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.960973 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.960259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgwpm\" (UniqueName: \"kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:49.990841 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:49.990806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:50.003672 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.003643 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x"] Jul 24 17:48:50.007414 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:48:50.007380 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5e1fdf_aa73_454e_9f5e_e834bbe764d9.slice/crio-22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7 WatchSource:0}: Error finding container 22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7: Status 404 returned error can't find the container with id 22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7 Jul 24 17:48:50.037044 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.037008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" event={"ID":"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9","Type":"ContainerStarted","Data":"22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7"} Jul 24 17:48:50.080709 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.080684 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp"] Jul 24 17:48:50.082429 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:48:50.082394 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a86554d_b9dc_4f19_acec_b44efae634be.slice/crio-b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf WatchSource:0}: Error finding container b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf: Status 404 returned error can't find the container with id b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf Jul 24 17:48:50.103639 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.103605 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:50.176867 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:48:50.176836 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23559a75_6ef8_4da8_b89a_c873d560cc53.slice/crio-1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b WatchSource:0}: Error finding container 1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b: Status 404 returned error can't find the container with id 1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b Jul 24 17:48:50.181233 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.181000 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl"] Jul 24 17:48:50.277719 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:50.277682 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml"] Jul 24 17:48:50.352267 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:48:50.352225 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127792f3_8027_45ca_b539_154db8c37cc2.slice/crio-7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6 WatchSource:0}: Error finding container 7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6: Status 404 returned error can't find the container with id 7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6 Jul 24 17:48:51.043568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.043528 2566 generic.go:356] "Generic (PLEG): container finished" podID="7a86554d-b9dc-4f19-acec-b44efae634be" containerID="80cf73546c247005a66ee73482c31dbeed0602c09c5419be836285e181d7a722" exitCode=0 Jul 24 17:48:51.044018 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.043618 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" event={"ID":"7a86554d-b9dc-4f19-acec-b44efae634be","Type":"ContainerDied","Data":"80cf73546c247005a66ee73482c31dbeed0602c09c5419be836285e181d7a722"} Jul 24 17:48:51.044018 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.043704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" event={"ID":"7a86554d-b9dc-4f19-acec-b44efae634be","Type":"ContainerStarted","Data":"b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf"} Jul 24 17:48:51.044751 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.044729 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 17:48:51.045189 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.045162 2566 generic.go:356] "Generic (PLEG): container finished" podID="127792f3-8027-45ca-b539-154db8c37cc2" containerID="d76d87109376d897cdd4f1e664d00a9859d19f7ed679ae31958941347d5bdb98" exitCode=0 Jul 24 17:48:51.045283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.045192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" event={"ID":"127792f3-8027-45ca-b539-154db8c37cc2","Type":"ContainerDied","Data":"d76d87109376d897cdd4f1e664d00a9859d19f7ed679ae31958941347d5bdb98"} Jul 24 17:48:51.045283 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.045239 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" event={"ID":"127792f3-8027-45ca-b539-154db8c37cc2","Type":"ContainerStarted","Data":"7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6"} Jul 24 17:48:51.046854 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.046832 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerID="42f096e4497e4fd0f92047ba852abb4dcaeb50b7cd7b6b9f0534eb890b15d203" exitCode=0 Jul 24 17:48:51.046942 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.046916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" event={"ID":"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9","Type":"ContainerDied","Data":"42f096e4497e4fd0f92047ba852abb4dcaeb50b7cd7b6b9f0534eb890b15d203"} Jul 24 17:48:51.048625 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.048598 2566 generic.go:356] "Generic (PLEG): container finished" podID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerID="0a628fba058b7e8201e150a0f73b82e5b50edc223743d5afe7e49ebe441949e3" exitCode=0 Jul 24 17:48:51.048702 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.048652 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" event={"ID":"23559a75-6ef8-4da8-b89a-c873d560cc53","Type":"ContainerDied","Data":"0a628fba058b7e8201e150a0f73b82e5b50edc223743d5afe7e49ebe441949e3"} Jul 24 17:48:51.048702 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:51.048683 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" event={"ID":"23559a75-6ef8-4da8-b89a-c873d560cc53","Type":"ContainerStarted","Data":"1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b"} Jul 24 17:48:52.055229 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.055138 2566 generic.go:356] "Generic (PLEG): container finished" podID="7a86554d-b9dc-4f19-acec-b44efae634be" containerID="4e3e9cb26a21504c0566b4d8ebf651fd0c17ece6f95ff314c737f52812e629af" exitCode=0 Jul 24 17:48:52.055704 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.055235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" event={"ID":"7a86554d-b9dc-4f19-acec-b44efae634be","Type":"ContainerDied","Data":"4e3e9cb26a21504c0566b4d8ebf651fd0c17ece6f95ff314c737f52812e629af"} Jul 24 17:48:52.057115 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.057088 2566 generic.go:356] "Generic (PLEG): container finished" podID="127792f3-8027-45ca-b539-154db8c37cc2" containerID="a30e8dc128ca80435d0c555dace81d90201a54e6a460c26e7a395beaa586cda5" exitCode=0 Jul 24 17:48:52.057217 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.057123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" event={"ID":"127792f3-8027-45ca-b539-154db8c37cc2","Type":"ContainerDied","Data":"a30e8dc128ca80435d0c555dace81d90201a54e6a460c26e7a395beaa586cda5"} Jul 24 17:48:52.059211 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.059189 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerID="e1f7e562db5cfe45d073e4161666eee7181be6465060630815f63dd78bef3308" exitCode=0 Jul 24 17:48:52.059275 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:52.059251 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" event={"ID":"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9","Type":"ContainerDied","Data":"e1f7e562db5cfe45d073e4161666eee7181be6465060630815f63dd78bef3308"} Jul 24 17:48:53.067000 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.066967 2566 generic.go:356] "Generic (PLEG): container finished" podID="7a86554d-b9dc-4f19-acec-b44efae634be" containerID="a7255e094b41937346f474b3d2201e8b9c4f60a33e680da66622331667360bb7" exitCode=0 Jul 24 17:48:53.067461 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.067060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" event={"ID":"7a86554d-b9dc-4f19-acec-b44efae634be","Type":"ContainerDied","Data":"a7255e094b41937346f474b3d2201e8b9c4f60a33e680da66622331667360bb7"} Jul 24 17:48:53.069140 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.069117 2566 generic.go:356] "Generic (PLEG): container finished" podID="127792f3-8027-45ca-b539-154db8c37cc2" containerID="3bb7b74fe183d12929c0f742d1fd6445a60877230844c3d1e888fb14d6740422" exitCode=0 Jul 24 17:48:53.069242 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.069198 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" event={"ID":"127792f3-8027-45ca-b539-154db8c37cc2","Type":"ContainerDied","Data":"3bb7b74fe183d12929c0f742d1fd6445a60877230844c3d1e888fb14d6740422"} Jul 24 17:48:53.071293 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.071272 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerID="0de83edbe6b548398d27ac6fa8c737c3758c021085790a3fe56b4f2037c881a2" exitCode=0 Jul 24 17:48:53.071412 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.071345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" event={"ID":"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9","Type":"ContainerDied","Data":"0de83edbe6b548398d27ac6fa8c737c3758c021085790a3fe56b4f2037c881a2"} Jul 24 17:48:53.073015 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.072997 2566 generic.go:356] "Generic (PLEG): container finished" podID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerID="8cefe6a9b1e5bfc963e72fc43f35c4bd4e557441cba3336e1244b72db221f128" exitCode=0 Jul 24 17:48:53.073086 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:53.073028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" event={"ID":"23559a75-6ef8-4da8-b89a-c873d560cc53","Type":"ContainerDied","Data":"8cefe6a9b1e5bfc963e72fc43f35c4bd4e557441cba3336e1244b72db221f128"} Jul 24 17:48:54.081603 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.081562 2566 generic.go:356] "Generic (PLEG): container finished" podID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerID="c23cff80df37c70e4582faa2a2acff648c07a5a83b386367f6ce8bb13c2b24ff" exitCode=0 Jul 24 17:48:54.082019 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.081604 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" event={"ID":"23559a75-6ef8-4da8-b89a-c873d560cc53","Type":"ContainerDied","Data":"c23cff80df37c70e4582faa2a2acff648c07a5a83b386367f6ce8bb13c2b24ff"} Jul 24 17:48:54.228155 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.228126 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:54.281096 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.281071 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:54.283898 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.283873 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:54.290548 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.290523 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle\") pod \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " Jul 24 17:48:54.290640 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.290627 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bkvj\" (UniqueName: \"kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj\") pod \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " Jul 24 17:48:54.290701 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.290673 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util\") pod \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\" (UID: \"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9\") " Jul 24 17:48:54.291260 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.291235 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle" (OuterVolumeSpecName: "bundle") pod "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" (UID: "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.292904 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.292881 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj" (OuterVolumeSpecName: "kube-api-access-2bkvj") pod "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" (UID: "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9"). InnerVolumeSpecName "kube-api-access-2bkvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:54.297103 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.297080 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util" (OuterVolumeSpecName: "util") pod "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" (UID: "0a5e1fdf-aa73-454e-9f5e-e834bbe764d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.392041 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util\") pod \"127792f3-8027-45ca-b539-154db8c37cc2\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " Jul 24 17:48:54.392254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392065 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle\") pod \"127792f3-8027-45ca-b539-154db8c37cc2\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " Jul 24 17:48:54.392254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util\") pod \"7a86554d-b9dc-4f19-acec-b44efae634be\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " Jul 24 17:48:54.392254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392146 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89mkj\" (UniqueName: \"kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj\") pod \"7a86554d-b9dc-4f19-acec-b44efae634be\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " Jul 24 17:48:54.392254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392172 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle\") pod \"7a86554d-b9dc-4f19-acec-b44efae634be\" (UID: \"7a86554d-b9dc-4f19-acec-b44efae634be\") " Jul 24 17:48:54.392254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392196 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgwpm\" (UniqueName: \"kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm\") pod \"127792f3-8027-45ca-b539-154db8c37cc2\" (UID: \"127792f3-8027-45ca-b539-154db8c37cc2\") " Jul 24 17:48:54.392717 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392686 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle" (OuterVolumeSpecName: "bundle") pod "127792f3-8027-45ca-b539-154db8c37cc2" (UID: "127792f3-8027-45ca-b539-154db8c37cc2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.392830 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.392697 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle" (OuterVolumeSpecName: "bundle") pod "7a86554d-b9dc-4f19-acec-b44efae634be" (UID: "7a86554d-b9dc-4f19-acec-b44efae634be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.393093 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.393075 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.393177 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.393098 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.393177 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.393111 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.393177 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.393121 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.393177 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.393136 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bkvj\" (UniqueName: \"kubernetes.io/projected/0a5e1fdf-aa73-454e-9f5e-e834bbe764d9-kube-api-access-2bkvj\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.394602 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.394579 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm" (OuterVolumeSpecName: "kube-api-access-kgwpm") pod "127792f3-8027-45ca-b539-154db8c37cc2" (UID: "127792f3-8027-45ca-b539-154db8c37cc2"). InnerVolumeSpecName "kube-api-access-kgwpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:54.394750 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.394728 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj" (OuterVolumeSpecName: "kube-api-access-89mkj") pod "7a86554d-b9dc-4f19-acec-b44efae634be" (UID: "7a86554d-b9dc-4f19-acec-b44efae634be"). InnerVolumeSpecName "kube-api-access-89mkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:54.397857 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.397833 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util" (OuterVolumeSpecName: "util") pod "127792f3-8027-45ca-b539-154db8c37cc2" (UID: "127792f3-8027-45ca-b539-154db8c37cc2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.398845 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.398824 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util" (OuterVolumeSpecName: "util") pod "7a86554d-b9dc-4f19-acec-b44efae634be" (UID: "7a86554d-b9dc-4f19-acec-b44efae634be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:54.493905 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.493862 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a86554d-b9dc-4f19-acec-b44efae634be-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.493905 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.493899 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89mkj\" (UniqueName: \"kubernetes.io/projected/7a86554d-b9dc-4f19-acec-b44efae634be-kube-api-access-89mkj\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.494101 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.493915 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgwpm\" (UniqueName: \"kubernetes.io/projected/127792f3-8027-45ca-b539-154db8c37cc2-kube-api-access-kgwpm\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:54.494101 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:54.493927 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/127792f3-8027-45ca-b539-154db8c37cc2-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:55.089115 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.089084 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" Jul 24 17:48:55.089115 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.089090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb7tzml" event={"ID":"127792f3-8027-45ca-b539-154db8c37cc2","Type":"ContainerDied","Data":"7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6"} Jul 24 17:48:55.089614 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.089140 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7784af2e7bea45a7b491a1649ebb53ccfba8328e768c6f54840a338823a165d6" Jul 24 17:48:55.091051 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.090991 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" Jul 24 17:48:55.091178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.090986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870539gl4x" event={"ID":"0a5e1fdf-aa73-454e-9f5e-e834bbe764d9","Type":"ContainerDied","Data":"22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7"} Jul 24 17:48:55.091178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.091117 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22b2591913ec3fcc66fc655b5fa129f74833f585cf84eae8efc027efa72a33f7" Jul 24 17:48:55.093139 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.093116 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" Jul 24 17:48:55.093264 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.093138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0hhxfp" event={"ID":"7a86554d-b9dc-4f19-acec-b44efae634be","Type":"ContainerDied","Data":"b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf"} Jul 24 17:48:55.093264 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.093179 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b289874147d28a06a65fe5d1bd29bc664c5c29e9aa16a99ebd4372e396c2ecaf" Jul 24 17:48:55.228703 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.228681 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:55.299863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.299830 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util\") pod \"23559a75-6ef8-4da8-b89a-c873d560cc53\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " Jul 24 17:48:55.299863 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.299871 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q89tk\" (UniqueName: \"kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk\") pod \"23559a75-6ef8-4da8-b89a-c873d560cc53\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " Jul 24 17:48:55.300082 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.299907 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle\") pod \"23559a75-6ef8-4da8-b89a-c873d560cc53\" (UID: \"23559a75-6ef8-4da8-b89a-c873d560cc53\") " Jul 24 17:48:55.300354 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.300330 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle" (OuterVolumeSpecName: "bundle") pod "23559a75-6ef8-4da8-b89a-c873d560cc53" (UID: "23559a75-6ef8-4da8-b89a-c873d560cc53"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:55.301939 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.301917 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk" (OuterVolumeSpecName: "kube-api-access-q89tk") pod "23559a75-6ef8-4da8-b89a-c873d560cc53" (UID: "23559a75-6ef8-4da8-b89a-c873d560cc53"). InnerVolumeSpecName "kube-api-access-q89tk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:48:55.305338 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.305303 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util" (OuterVolumeSpecName: "util") pod "23559a75-6ef8-4da8-b89a-c873d560cc53" (UID: "23559a75-6ef8-4da8-b89a-c873d560cc53"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 17:48:55.401280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.401243 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-util\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:55.401280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.401274 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q89tk\" (UniqueName: \"kubernetes.io/projected/23559a75-6ef8-4da8-b89a-c873d560cc53-kube-api-access-q89tk\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:55.401280 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:55.401286 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23559a75-6ef8-4da8-b89a-c873d560cc53-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:48:56.099428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:56.099388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" event={"ID":"23559a75-6ef8-4da8-b89a-c873d560cc53","Type":"ContainerDied","Data":"1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b"} Jul 24 17:48:56.099428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:56.099425 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c56ksl" Jul 24 17:48:56.099874 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:48:56.099447 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ba00877a2f328cd1ffc9ea9abeeb71b061e63a10c0441803757abdb7e17ec0b" Jul 24 17:49:18.912368 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912319 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr"] Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912684 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912699 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912709 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912724 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912730 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912735 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912741 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912748 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912754 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912759 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912764 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912770 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912778 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912783 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912792 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912797 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="extract" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912809 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912813 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912819 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912824 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="pull" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912831 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912836 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912847 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="util" Jul 24 17:49:18.912869 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912855 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="util" Jul 24 17:49:18.913697 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912905 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5e1fdf-aa73-454e-9f5e-e834bbe764d9" containerName="extract" Jul 24 17:49:18.913697 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912914 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="127792f3-8027-45ca-b539-154db8c37cc2" containerName="extract" Jul 24 17:49:18.913697 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912920 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="23559a75-6ef8-4da8-b89a-c873d560cc53" containerName="extract" Jul 24 17:49:18.913697 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.912926 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a86554d-b9dc-4f19-acec-b44efae634be" containerName="extract" Jul 24 17:49:18.915751 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.915732 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:18.918402 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.918375 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 17:49:18.918402 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.918387 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 17:49:18.918598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.918392 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 17:49:18.919424 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.919406 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kfgrn\"" Jul 24 17:49:18.919538 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.919446 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 17:49:18.923595 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:18.923564 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr"] Jul 24 17:49:19.005686 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.005646 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7639a455-08ce-4430-ae43-e3b73952f11e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.005686 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.005689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7639a455-08ce-4430-ae43-e3b73952f11e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.005926 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.005724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5sqc\" (UniqueName: \"kubernetes.io/projected/7639a455-08ce-4430-ae43-e3b73952f11e-kube-api-access-f5sqc\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.106457 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.106417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7639a455-08ce-4430-ae43-e3b73952f11e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.106457 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.106459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7639a455-08ce-4430-ae43-e3b73952f11e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.106673 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.106503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5sqc\" (UniqueName: \"kubernetes.io/projected/7639a455-08ce-4430-ae43-e3b73952f11e-kube-api-access-f5sqc\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.107070 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.107046 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7639a455-08ce-4430-ae43-e3b73952f11e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.108953 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.108930 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7639a455-08ce-4430-ae43-e3b73952f11e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.115587 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.115561 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5sqc\" (UniqueName: \"kubernetes.io/projected/7639a455-08ce-4430-ae43-e3b73952f11e-kube-api-access-f5sqc\") pod \"kuadrant-console-plugin-598f86468b-4kczr\" (UID: \"7639a455-08ce-4430-ae43-e3b73952f11e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.225672 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.225592 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" Jul 24 17:49:19.354580 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:19.354547 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr"] Jul 24 17:49:19.357457 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:49:19.357427 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7639a455_08ce_4430_ae43_e3b73952f11e.slice/crio-836374a20e91271a0a0802fae34f6bab9026a018b215794ebd383fc7b82b460c WatchSource:0}: Error finding container 836374a20e91271a0a0802fae34f6bab9026a018b215794ebd383fc7b82b460c: Status 404 returned error can't find the container with id 836374a20e91271a0a0802fae34f6bab9026a018b215794ebd383fc7b82b460c Jul 24 17:49:20.221657 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:20.221620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" event={"ID":"7639a455-08ce-4430-ae43-e3b73952f11e","Type":"ContainerStarted","Data":"836374a20e91271a0a0802fae34f6bab9026a018b215794ebd383fc7b82b460c"} Jul 24 17:49:24.248674 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:24.248637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" event={"ID":"7639a455-08ce-4430-ae43-e3b73952f11e","Type":"ContainerStarted","Data":"665261313b16039833df4a499cd5461199f8e1067c61d2ca224bf81d4a758bfe"} Jul 24 17:49:24.263062 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:24.263008 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-4kczr" podStartSLOduration=1.533727001 podStartE2EDuration="6.262991468s" totalImagesPullingTime="4.729264467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:24.262410807 +0000 UTC m=+380.463643197" watchObservedRunningTime="2026-07-24 17:49:24.262991468 +0000 UTC m=+380.464223851" Jul 24 17:49:54.558906 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.558867 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65d6879f4f-qsbqw"] Jul 24 17:49:54.561428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.561404 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.575526 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.575500 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d6879f4f-qsbqw"] Jul 24 17:49:54.604090 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-service-ca\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604090 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-oauth-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-oauth-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604322 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-trusted-ca-bundle\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604404 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-console-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.604578 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.604429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97fvp\" (UniqueName: \"kubernetes.io/projected/06a182a9-b09b-40be-9506-983ed758ed97-kube-api-access-97fvp\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705393 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-trusted-ca-bundle\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705393 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-console-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97fvp\" (UniqueName: \"kubernetes.io/projected/06a182a9-b09b-40be-9506-983ed758ed97-kube-api-access-97fvp\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705441 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-service-ca\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-oauth-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.705631 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.705524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-oauth-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.706266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.706234 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-console-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.706266 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.706258 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-trusted-ca-bundle\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.706453 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.706240 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-oauth-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.706453 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.706321 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06a182a9-b09b-40be-9506-983ed758ed97-service-ca\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.708052 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.708024 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-oauth-config\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.708196 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.708178 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06a182a9-b09b-40be-9506-983ed758ed97-console-serving-cert\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.713875 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.713854 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97fvp\" (UniqueName: \"kubernetes.io/projected/06a182a9-b09b-40be-9506-983ed758ed97-kube-api-access-97fvp\") pod \"console-65d6879f4f-qsbqw\" (UID: \"06a182a9-b09b-40be-9506-983ed758ed97\") " pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:54.870937 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:54.870894 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:49:55.011413 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:55.011387 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d6879f4f-qsbqw"] Jul 24 17:49:55.013138 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:49:55.013103 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a182a9_b09b_40be_9506_983ed758ed97.slice/crio-22ec1c9c03512c1c5a59072d8307ca56205995a5464b054336a841ecb2dab27d WatchSource:0}: Error finding container 22ec1c9c03512c1c5a59072d8307ca56205995a5464b054336a841ecb2dab27d: Status 404 returned error can't find the container with id 22ec1c9c03512c1c5a59072d8307ca56205995a5464b054336a841ecb2dab27d Jul 24 17:49:55.402732 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:55.402700 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d6879f4f-qsbqw" event={"ID":"06a182a9-b09b-40be-9506-983ed758ed97","Type":"ContainerStarted","Data":"889405323c1bc771e28b09bff8cc26367427e57c78c1f04b6c0bfe7ea7cdc182"} Jul 24 17:49:55.402900 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:55.402751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d6879f4f-qsbqw" event={"ID":"06a182a9-b09b-40be-9506-983ed758ed97","Type":"ContainerStarted","Data":"22ec1c9c03512c1c5a59072d8307ca56205995a5464b054336a841ecb2dab27d"} Jul 24 17:49:55.434477 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:49:55.434411 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65d6879f4f-qsbqw" podStartSLOduration=1.43439533 podStartE2EDuration="1.43439533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:55.433067399 +0000 UTC m=+411.634299837" watchObservedRunningTime="2026-07-24 17:49:55.43439533 +0000 UTC m=+411.635627722" Jul 24 17:50:04.871986 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:04.871945 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:50:04.872444 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:04.872031 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:50:04.877035 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:04.877008 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:50:05.459241 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:05.459213 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65d6879f4f-qsbqw" Jul 24 17:50:05.508978 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:05.508942 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:50:14.268817 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.268775 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:14.272189 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.272160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:14.274954 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.274931 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mxjvz\"" Jul 24 17:50:14.280286 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.280255 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:14.380185 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.380137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79ktn\" (UniqueName: \"kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn\") pod \"authorino-b54df94b4-ssbkx\" (UID: \"eaf4eafd-a490-4da3-82c3-ff5cb251618b\") " pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:14.415335 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.415295 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:14.418441 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.418424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:14.424022 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.423992 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:14.481385 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.481319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcf62\" (UniqueName: \"kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62\") pod \"authorino-c548b5d7d-8kh4l\" (UID: \"4afcd65e-5c15-49c7-b269-90a9efe6c7ed\") " pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:14.481566 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.481436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79ktn\" (UniqueName: \"kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn\") pod \"authorino-b54df94b4-ssbkx\" (UID: \"eaf4eafd-a490-4da3-82c3-ff5cb251618b\") " pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:14.490057 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.490016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79ktn\" (UniqueName: \"kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn\") pod \"authorino-b54df94b4-ssbkx\" (UID: \"eaf4eafd-a490-4da3-82c3-ff5cb251618b\") " pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:14.582679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.582594 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:14.582955 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.582932 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcf62\" (UniqueName: \"kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62\") pod \"authorino-c548b5d7d-8kh4l\" (UID: \"4afcd65e-5c15-49c7-b269-90a9efe6c7ed\") " pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:14.591665 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.591638 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcf62\" (UniqueName: \"kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62\") pod \"authorino-c548b5d7d-8kh4l\" (UID: \"4afcd65e-5c15-49c7-b269-90a9efe6c7ed\") " pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:14.718615 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.718589 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:14.721132 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:50:14.721101 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeaf4eafd_a490_4da3_82c3_ff5cb251618b.slice/crio-5597c60a8830d5a8355d4d8c95ad29be56e90930182f97b450ac1dd6ad0fb2cf WatchSource:0}: Error finding container 5597c60a8830d5a8355d4d8c95ad29be56e90930182f97b450ac1dd6ad0fb2cf: Status 404 returned error can't find the container with id 5597c60a8830d5a8355d4d8c95ad29be56e90930182f97b450ac1dd6ad0fb2cf Jul 24 17:50:14.728792 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.728769 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:14.861419 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:14.861388 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:14.862751 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:50:14.862724 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4afcd65e_5c15_49c7_b269_90a9efe6c7ed.slice/crio-89a8d8c1003a49ea169c399e8b1182f76b34149f389c3eebcc82a85358667cbf WatchSource:0}: Error finding container 89a8d8c1003a49ea169c399e8b1182f76b34149f389c3eebcc82a85358667cbf: Status 404 returned error can't find the container with id 89a8d8c1003a49ea169c399e8b1182f76b34149f389c3eebcc82a85358667cbf Jul 24 17:50:15.509314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:15.509272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" event={"ID":"4afcd65e-5c15-49c7-b269-90a9efe6c7ed","Type":"ContainerStarted","Data":"89a8d8c1003a49ea169c399e8b1182f76b34149f389c3eebcc82a85358667cbf"} Jul 24 17:50:15.511333 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:15.511293 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ssbkx" event={"ID":"eaf4eafd-a490-4da3-82c3-ff5cb251618b","Type":"ContainerStarted","Data":"5597c60a8830d5a8355d4d8c95ad29be56e90930182f97b450ac1dd6ad0fb2cf"} Jul 24 17:50:18.531652 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:18.531614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" event={"ID":"4afcd65e-5c15-49c7-b269-90a9efe6c7ed","Type":"ContainerStarted","Data":"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7"} Jul 24 17:50:18.533128 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:18.533102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ssbkx" event={"ID":"eaf4eafd-a490-4da3-82c3-ff5cb251618b","Type":"ContainerStarted","Data":"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f"} Jul 24 17:50:18.549515 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:18.549457 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" podStartSLOduration=1.870633268 podStartE2EDuration="4.549440051s" totalImagesPullingTime="2.678806783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:18.547165164 +0000 UTC m=+434.748397553" watchObservedRunningTime="2026-07-24 17:50:18.549440051 +0000 UTC m=+434.750672434" Jul 24 17:50:18.564796 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:18.564745 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-ssbkx" podStartSLOduration=1.755948316 podStartE2EDuration="4.564729693s" totalImagesPullingTime="2.808781377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:18.563380157 +0000 UTC m=+434.764612550" watchObservedRunningTime="2026-07-24 17:50:18.564729693 +0000 UTC m=+434.765962082" Jul 24 17:50:18.593031 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:18.592992 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:20.542261 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:20.542216 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-ssbkx" podUID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" containerName="authorino" containerID="cri-o://8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f" gracePeriod=30 Jul 24 17:50:20.783866 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:20.783838 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:20.942909 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:20.942851 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79ktn\" (UniqueName: \"kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn\") pod \"eaf4eafd-a490-4da3-82c3-ff5cb251618b\" (UID: \"eaf4eafd-a490-4da3-82c3-ff5cb251618b\") " Jul 24 17:50:20.945196 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:20.945159 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn" (OuterVolumeSpecName: "kube-api-access-79ktn") pod "eaf4eafd-a490-4da3-82c3-ff5cb251618b" (UID: "eaf4eafd-a490-4da3-82c3-ff5cb251618b"). InnerVolumeSpecName "kube-api-access-79ktn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:50:21.044096 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.044054 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79ktn\" (UniqueName: \"kubernetes.io/projected/eaf4eafd-a490-4da3-82c3-ff5cb251618b-kube-api-access-79ktn\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:21.548416 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.548342 2566 generic.go:356] "Generic (PLEG): container finished" podID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" containerID="8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f" exitCode=0 Jul 24 17:50:21.548416 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.548389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ssbkx" event={"ID":"eaf4eafd-a490-4da3-82c3-ff5cb251618b","Type":"ContainerDied","Data":"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f"} Jul 24 17:50:21.548979 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.548420 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-ssbkx" Jul 24 17:50:21.548979 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.548434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-ssbkx" event={"ID":"eaf4eafd-a490-4da3-82c3-ff5cb251618b","Type":"ContainerDied","Data":"5597c60a8830d5a8355d4d8c95ad29be56e90930182f97b450ac1dd6ad0fb2cf"} Jul 24 17:50:21.548979 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.548454 2566 scope.go:117] "RemoveContainer" containerID="8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f" Jul 24 17:50:21.557900 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.557881 2566 scope.go:117] "RemoveContainer" containerID="8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f" Jul 24 17:50:21.558171 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:50:21.558151 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f\": container with ID starting with 8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f not found: ID does not exist" containerID="8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f" Jul 24 17:50:21.558221 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.558183 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f"} err="failed to get container status \"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f\": rpc error: code = NotFound desc = could not find container \"8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f\": container with ID starting with 8c243fb13698d7a1e55dc13f7c247c852bacde51be3fee09a7ae84a9c54c943f not found: ID does not exist" Jul 24 17:50:21.570110 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.570067 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:21.573315 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:21.573282 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-ssbkx"] Jul 24 17:50:22.250181 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:22.250143 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" path="/var/lib/kubelet/pods/eaf4eafd-a490-4da3-82c3-ff5cb251618b/volumes" Jul 24 17:50:30.528446 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.528339 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b69b97dff-4x4kq" podUID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" containerName="console" containerID="cri-o://cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86" gracePeriod=15 Jul 24 17:50:30.774683 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.774655 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:50:30.929472 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929519 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929541 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929560 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929587 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8kth\" (UniqueName: \"kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929604 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929618 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca\") pod \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\" (UID: \"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e\") " Jul 24 17:50:30.929990 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929956 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:50:30.930046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.929999 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config" (OuterVolumeSpecName: "console-config") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:50:30.930046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.930011 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:50:30.930235 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.930195 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca" (OuterVolumeSpecName: "service-ca") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 17:50:30.931857 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.931834 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:50:30.931953 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.931859 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:50:30.932033 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:30.932015 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth" (OuterVolumeSpecName: "kube-api-access-r8kth") pod "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" (UID: "16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e"). InnerVolumeSpecName "kube-api-access-r8kth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:50:31.030323 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030284 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030323 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030318 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030323 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030330 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-oauth-serving-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030585 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030339 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8kth\" (UniqueName: \"kubernetes.io/projected/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-kube-api-access-r8kth\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030585 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030348 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-console-oauth-config\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030585 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030356 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.030585 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.030376 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e-trusted-ca-bundle\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:31.603351 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.603319 2566 generic.go:356] "Generic (PLEG): container finished" podID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" containerID="cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86" exitCode=0 Jul 24 17:50:31.603835 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.603387 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b69b97dff-4x4kq" Jul 24 17:50:31.603835 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.603403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69b97dff-4x4kq" event={"ID":"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e","Type":"ContainerDied","Data":"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86"} Jul 24 17:50:31.603835 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.603467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b69b97dff-4x4kq" event={"ID":"16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e","Type":"ContainerDied","Data":"fa48a80a608c33ad3a9a39822752663d776fa1858914635f3793012e688c10c6"} Jul 24 17:50:31.603835 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.603497 2566 scope.go:117] "RemoveContainer" containerID="cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86" Jul 24 17:50:31.613389 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.613349 2566 scope.go:117] "RemoveContainer" containerID="cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86" Jul 24 17:50:31.613652 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:50:31.613634 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86\": container with ID starting with cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86 not found: ID does not exist" containerID="cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86" Jul 24 17:50:31.613709 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.613662 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86"} err="failed to get container status \"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86\": rpc error: code = NotFound desc = could not find container \"cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86\": container with ID starting with cf980d3c273fb32100be5a2727fd58de13742f754fa11a658330794522861a86 not found: ID does not exist" Jul 24 17:50:31.628461 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.628413 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:50:31.633284 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:31.633259 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b69b97dff-4x4kq"] Jul 24 17:50:32.255156 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:32.255121 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" path="/var/lib/kubelet/pods/16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e/volumes" Jul 24 17:50:40.602677 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.602644 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-lwn5n"] Jul 24 17:50:40.603133 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603116 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" containerName="console" Jul 24 17:50:40.603200 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603136 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" containerName="console" Jul 24 17:50:40.603252 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603199 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" containerName="authorino" Jul 24 17:50:40.603252 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603209 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" containerName="authorino" Jul 24 17:50:40.603348 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603302 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf4eafd-a490-4da3-82c3-ff5cb251618b" containerName="authorino" Jul 24 17:50:40.603348 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.603320 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="16cafddd-3065-4dba-bc3a-e7f7a9ba3d1e" containerName="console" Jul 24 17:50:40.607590 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.607568 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:40.614508 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.614479 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-lwn5n"] Jul 24 17:50:40.677067 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.677017 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-lwn5n"] Jul 24 17:50:40.677309 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:50:40.677289 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cgmxr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" podUID="22e7b75c-1c34-4a5b-8cca-3dd49986ff42" Jul 24 17:50:40.708709 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.708668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgmxr\" (UniqueName: \"kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr\") pod \"authorino-5dc7ccb5f-lwn5n\" (UID: \"22e7b75c-1c34-4a5b-8cca-3dd49986ff42\") " pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:40.810118 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.810082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgmxr\" (UniqueName: \"kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr\") pod \"authorino-5dc7ccb5f-lwn5n\" (UID: \"22e7b75c-1c34-4a5b-8cca-3dd49986ff42\") " pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:40.818825 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.818792 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgmxr\" (UniqueName: \"kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr\") pod \"authorino-5dc7ccb5f-lwn5n\" (UID: \"22e7b75c-1c34-4a5b-8cca-3dd49986ff42\") " pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:40.906094 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.906007 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:50:40.909405 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.909384 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:40.912409 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.912385 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 17:50:40.921165 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:40.921134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:50:41.011721 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.011689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.011721 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.011727 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7sfc\" (UniqueName: \"kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.112518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.112481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.112518 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.112520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7sfc\" (UniqueName: \"kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.114911 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.114890 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.121322 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.121297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7sfc\" (UniqueName: \"kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc\") pod \"authorino-86d6d5d896-hldh9\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.219354 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.219266 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:50:41.349349 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.349318 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:50:41.350817 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:50:41.350793 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65946d8b_e317_4b86_a712_74fd9efcdaa6.slice/crio-7fb11c282d18f5e0c8225357388e82bdff74fa38acab4ad0450399f7e241c5c2 WatchSource:0}: Error finding container 7fb11c282d18f5e0c8225357388e82bdff74fa38acab4ad0450399f7e241c5c2: Status 404 returned error can't find the container with id 7fb11c282d18f5e0c8225357388e82bdff74fa38acab4ad0450399f7e241c5c2 Jul 24 17:50:41.663561 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.663524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86d6d5d896-hldh9" event={"ID":"65946d8b-e317-4b86-a712-74fd9efcdaa6","Type":"ContainerStarted","Data":"7fb11c282d18f5e0c8225357388e82bdff74fa38acab4ad0450399f7e241c5c2"} Jul 24 17:50:41.664020 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.663605 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:41.671643 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.671621 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:41.716079 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.716056 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgmxr\" (UniqueName: \"kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr\") pod \"22e7b75c-1c34-4a5b-8cca-3dd49986ff42\" (UID: \"22e7b75c-1c34-4a5b-8cca-3dd49986ff42\") " Jul 24 17:50:41.718046 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.718025 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr" (OuterVolumeSpecName: "kube-api-access-cgmxr") pod "22e7b75c-1c34-4a5b-8cca-3dd49986ff42" (UID: "22e7b75c-1c34-4a5b-8cca-3dd49986ff42"). InnerVolumeSpecName "kube-api-access-cgmxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:50:41.817264 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:41.817234 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgmxr\" (UniqueName: \"kubernetes.io/projected/22e7b75c-1c34-4a5b-8cca-3dd49986ff42-kube-api-access-cgmxr\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:42.669722 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.669668 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86d6d5d896-hldh9" event={"ID":"65946d8b-e317-4b86-a712-74fd9efcdaa6","Type":"ContainerStarted","Data":"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5"} Jul 24 17:50:42.669722 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.669683 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-lwn5n" Jul 24 17:50:42.685620 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.685569 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86d6d5d896-hldh9" podStartSLOduration=2.192214735 podStartE2EDuration="2.685554311s" totalImagesPullingTime="493.339576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:50:42.683632276 +0000 UTC m=+458.884864664" watchObservedRunningTime="2026-07-24 17:50:42.685554311 +0000 UTC m=+458.886786699" Jul 24 17:50:42.708085 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.708051 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-lwn5n"] Jul 24 17:50:42.712412 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.712385 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:42.712598 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.712576 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" podUID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" containerName="authorino" containerID="cri-o://41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7" gracePeriod=30 Jul 24 17:50:42.717254 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.717216 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-lwn5n"] Jul 24 17:50:42.959042 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:42.959018 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:43.126001 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.125960 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcf62\" (UniqueName: \"kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62\") pod \"4afcd65e-5c15-49c7-b269-90a9efe6c7ed\" (UID: \"4afcd65e-5c15-49c7-b269-90a9efe6c7ed\") " Jul 24 17:50:43.127941 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.127903 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62" (OuterVolumeSpecName: "kube-api-access-pcf62") pod "4afcd65e-5c15-49c7-b269-90a9efe6c7ed" (UID: "4afcd65e-5c15-49c7-b269-90a9efe6c7ed"). InnerVolumeSpecName "kube-api-access-pcf62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:50:43.227148 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.227117 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcf62\" (UniqueName: \"kubernetes.io/projected/4afcd65e-5c15-49c7-b269-90a9efe6c7ed-kube-api-access-pcf62\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:50:43.675931 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.675895 2566 generic.go:356] "Generic (PLEG): container finished" podID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" containerID="41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7" exitCode=0 Jul 24 17:50:43.676340 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.675950 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" Jul 24 17:50:43.676340 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.675979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" event={"ID":"4afcd65e-5c15-49c7-b269-90a9efe6c7ed","Type":"ContainerDied","Data":"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7"} Jul 24 17:50:43.676340 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.676026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-8kh4l" event={"ID":"4afcd65e-5c15-49c7-b269-90a9efe6c7ed","Type":"ContainerDied","Data":"89a8d8c1003a49ea169c399e8b1182f76b34149f389c3eebcc82a85358667cbf"} Jul 24 17:50:43.676340 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.676043 2566 scope.go:117] "RemoveContainer" containerID="41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7" Jul 24 17:50:43.684888 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.684871 2566 scope.go:117] "RemoveContainer" containerID="41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7" Jul 24 17:50:43.685169 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:50:43.685151 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7\": container with ID starting with 41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7 not found: ID does not exist" containerID="41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7" Jul 24 17:50:43.685219 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.685177 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7"} err="failed to get container status \"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7\": rpc error: code = NotFound desc = could not find container \"41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7\": container with ID starting with 41af7a17c6c34c11aa83531504b988ed92885e6da5b037ae573736a4f14055c7 not found: ID does not exist" Jul 24 17:50:43.699047 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.699008 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:43.700879 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:43.700861 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-8kh4l"] Jul 24 17:50:44.248783 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:44.248751 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22e7b75c-1c34-4a5b-8cca-3dd49986ff42" path="/var/lib/kubelet/pods/22e7b75c-1c34-4a5b-8cca-3dd49986ff42/volumes" Jul 24 17:50:44.248983 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:50:44.248971 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" path="/var/lib/kubelet/pods/4afcd65e-5c15-49c7-b269-90a9efe6c7ed/volumes" Jul 24 17:51:23.246506 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.246466 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf"] Jul 24 17:51:23.246920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.246859 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" containerName="authorino" Jul 24 17:51:23.246920 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.246870 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" containerName="authorino" Jul 24 17:51:23.246992 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.246938 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4afcd65e-5c15-49c7-b269-90a9efe6c7ed" containerName="authorino" Jul 24 17:51:23.251348 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.251325 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.254161 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.254132 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2k9vq\"" Jul 24 17:51:23.254161 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.254151 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 17:51:23.261294 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.261268 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf"] Jul 24 17:51:23.384493 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.384454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njfs\" (UniqueName: \"kubernetes.io/projected/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-kube-api-access-4njfs\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.384675 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.384511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.485802 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.485766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njfs\" (UniqueName: \"kubernetes.io/projected/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-kube-api-access-4njfs\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.485977 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.485817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.486526 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.486508 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.493763 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.493742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njfs\" (UniqueName: \"kubernetes.io/projected/2626cd9e-d17e-43fa-a7cf-3a18ce16848e-kube-api-access-4njfs\") pod \"payload-pre-processing-687dbb6cf6-tj4qf\" (UID: \"2626cd9e-d17e-43fa-a7cf-3a18ce16848e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.562875 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.562790 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:23.700812 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.700779 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf"] Jul 24 17:51:23.702963 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:51:23.702934 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2626cd9e_d17e_43fa_a7cf_3a18ce16848e.slice/crio-ad3504634d5a7557d13c5f4a3d34a260ee766ba012b7d9ecb1b6815e647c7b2d WatchSource:0}: Error finding container ad3504634d5a7557d13c5f4a3d34a260ee766ba012b7d9ecb1b6815e647c7b2d: Status 404 returned error can't find the container with id ad3504634d5a7557d13c5f4a3d34a260ee766ba012b7d9ecb1b6815e647c7b2d Jul 24 17:51:23.876355 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:23.876318 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" event={"ID":"2626cd9e-d17e-43fa-a7cf-3a18ce16848e","Type":"ContainerStarted","Data":"ad3504634d5a7557d13c5f4a3d34a260ee766ba012b7d9ecb1b6815e647c7b2d"} Jul 24 17:51:25.888372 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:25.888327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" event={"ID":"2626cd9e-d17e-43fa-a7cf-3a18ce16848e","Type":"ContainerStarted","Data":"e991c297c4c15875bc6c229b4aad921db34694507cfe37b3b3c6e2fe34dad5df"} Jul 24 17:51:25.888790 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:25.888496 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:51:25.911278 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:25.911221 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" podStartSLOduration=1.491960503 podStartE2EDuration="2.911201012s" totalImagesPullingTime="1.419240509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:51:25.908470643 +0000 UTC m=+502.109703055" watchObservedRunningTime="2026-07-24 17:51:25.911201012 +0000 UTC m=+502.112433448" Jul 24 17:51:36.895933 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:51:36.895902 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-tj4qf" Jul 24 17:52:00.976192 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.976088 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp"] Jul 24 17:52:00.981075 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.981051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:00.984180 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.984161 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vxck9\"" Jul 24 17:52:00.985199 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.985169 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 17:52:00.985319 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.985244 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 17:52:00.985319 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.985183 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 17:52:00.990427 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:00.990405 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp"] Jul 24 17:52:01.110806 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.110767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.110976 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.110882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a3d9c72-020f-401e-b03f-e61103d61f60-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.110976 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.110936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.110976 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.110972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.111080 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.110986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.111080 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.111008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb8vm\" (UniqueName: \"kubernetes.io/projected/5a3d9c72-020f-401e-b03f-e61103d61f60-kube-api-access-rb8vm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.211796 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.211758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.211796 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.211800 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.211821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.211898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb8vm\" (UniqueName: \"kubernetes.io/projected/5a3d9c72-020f-401e-b03f-e61103d61f60-kube-api-access-rb8vm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.211960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212084 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.212070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a3d9c72-020f-401e-b03f-e61103d61f60-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.212194 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.212233 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.212296 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.212264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.214276 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.214251 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a3d9c72-020f-401e-b03f-e61103d61f60-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.214587 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.214556 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a3d9c72-020f-401e-b03f-e61103d61f60-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.222690 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.222665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb8vm\" (UniqueName: \"kubernetes.io/projected/5a3d9c72-020f-401e-b03f-e61103d61f60-kube-api-access-rb8vm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp\" (UID: \"5a3d9c72-020f-401e-b03f-e61103d61f60\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.291724 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.291634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:01.429846 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:01.429818 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp"] Jul 24 17:52:01.430833 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:52:01.430807 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3d9c72_020f_401e_b03f_e61103d61f60.slice/crio-c3ec0c35aa8d835be5e98a3d90b6f2ac410118487314be2ea66da0857582be52 WatchSource:0}: Error finding container c3ec0c35aa8d835be5e98a3d90b6f2ac410118487314be2ea66da0857582be52: Status 404 returned error can't find the container with id c3ec0c35aa8d835be5e98a3d90b6f2ac410118487314be2ea66da0857582be52 Jul 24 17:52:02.076513 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:02.076479 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" event={"ID":"5a3d9c72-020f-401e-b03f-e61103d61f60","Type":"ContainerStarted","Data":"c3ec0c35aa8d835be5e98a3d90b6f2ac410118487314be2ea66da0857582be52"} Jul 24 17:52:03.083467 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:03.083428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" event={"ID":"5a3d9c72-020f-401e-b03f-e61103d61f60","Type":"ContainerStarted","Data":"8fcbf4588b7b2babb33790ec5ea848a32c2a98fd51a44f1036363366f02baab4"} Jul 24 17:52:11.291989 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:11.291955 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:11.292559 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:11.292054 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:11.305309 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:11.305284 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:11.326710 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:11.326651 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" podStartSLOduration=10.305316721 podStartE2EDuration="11.326633104s" totalImagesPullingTime="1.021316383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:52:03.105453979 +0000 UTC m=+539.306686370" watchObservedRunningTime="2026-07-24 17:52:11.326633104 +0000 UTC m=+547.527865494" Jul 24 17:52:12.139912 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:12.139878 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp" Jul 24 17:52:42.739193 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.739160 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g"] Jul 24 17:52:42.741672 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.741653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.744299 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.744277 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 17:52:42.755064 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.755038 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g"] Jul 24 17:52:42.853385 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853332 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.853385 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.853620 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brljh\" (UniqueName: \"kubernetes.io/projected/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-kube-api-access-brljh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.853620 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.853620 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.853724 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.853613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954288 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954288 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954342 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brljh\" (UniqueName: \"kubernetes.io/projected/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-kube-api-access-brljh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954568 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954781 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954850 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954810 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.954915 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.954874 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.956718 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.956695 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.956894 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.956875 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:42.962679 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:42.962656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brljh\" (UniqueName: \"kubernetes.io/projected/d7ebfd30-99fb-4e40-b6d1-d2b09e16a455-kube-api-access-brljh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47q5g\" (UID: \"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:43.052433 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:43.052305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:43.186318 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:43.186292 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g"] Jul 24 17:52:43.188672 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:52:43.188625 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ebfd30_99fb_4e40_b6d1_d2b09e16a455.slice/crio-bddf9608b68e60efc57c8f6d52f90dfef381260a7d08fc3dfeefa36dc2899902 WatchSource:0}: Error finding container bddf9608b68e60efc57c8f6d52f90dfef381260a7d08fc3dfeefa36dc2899902: Status 404 returned error can't find the container with id bddf9608b68e60efc57c8f6d52f90dfef381260a7d08fc3dfeefa36dc2899902 Jul 24 17:52:43.282589 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:43.282555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" event={"ID":"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455","Type":"ContainerStarted","Data":"bddf9608b68e60efc57c8f6d52f90dfef381260a7d08fc3dfeefa36dc2899902"} Jul 24 17:52:44.289178 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:44.289140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" event={"ID":"d7ebfd30-99fb-4e40-b6d1-d2b09e16a455","Type":"ContainerStarted","Data":"9ed8b309871f41462023f6a68e7cf5b697588c2b6c193afc3195e62cc1eb8591"} Jul 24 17:52:44.333893 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:44.333835 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" podStartSLOduration=2.165274243 podStartE2EDuration="2.333818704s" totalImagesPullingTime="168.544461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:52:44.333063418 +0000 UTC m=+580.534295806" watchObservedRunningTime="2026-07-24 17:52:44.333818704 +0000 UTC m=+580.535051093" Jul 24 17:52:53.053406 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:53.053345 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:53.053795 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:53.053427 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:53.065742 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:53.065720 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:52:53.349188 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:52:53.349097 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47q5g" Jul 24 17:53:02.355611 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.355570 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx"] Jul 24 17:53:02.358104 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.358086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.361013 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.360981 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 24 17:53:02.370698 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.370668 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx"] Jul 24 17:53:02.426783 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4821ddfb-f57a-4667-b34f-015b070b9bf3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.426969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.426969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.426969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.426969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvfl7\" (UniqueName: \"kubernetes.io/projected/4821ddfb-f57a-4667-b34f-015b070b9bf3-kube-api-access-dvfl7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.426969 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.426952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528241 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvfl7\" (UniqueName: \"kubernetes.io/projected/4821ddfb-f57a-4667-b34f-015b070b9bf3-kube-api-access-dvfl7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528445 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528654 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4821ddfb-f57a-4667-b34f-015b070b9bf3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528654 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528649 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528777 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528723 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.528838 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.528808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.530528 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.530499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4821ddfb-f57a-4667-b34f-015b070b9bf3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.530937 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.530919 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4821ddfb-f57a-4667-b34f-015b070b9bf3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.536782 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.536763 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvfl7\" (UniqueName: \"kubernetes.io/projected/4821ddfb-f57a-4667-b34f-015b070b9bf3-kube-api-access-dvfl7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx\" (UID: \"4821ddfb-f57a-4667-b34f-015b070b9bf3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.668053 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.667961 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:02.803606 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:02.803580 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx"] Jul 24 17:53:02.805817 ip-10-0-132-71 kubenswrapper[2566]: W0724 17:53:02.805793 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4821ddfb_f57a_4667_b34f_015b070b9bf3.slice/crio-ef45101bd6aabc8817a262f9c36a8193eb3fb59542f74a6925ce3be5a9fa1a4c WatchSource:0}: Error finding container ef45101bd6aabc8817a262f9c36a8193eb3fb59542f74a6925ce3be5a9fa1a4c: Status 404 returned error can't find the container with id ef45101bd6aabc8817a262f9c36a8193eb3fb59542f74a6925ce3be5a9fa1a4c Jul 24 17:53:03.391170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:03.391133 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" event={"ID":"4821ddfb-f57a-4667-b34f-015b070b9bf3","Type":"ContainerStarted","Data":"2a036365fd2d29e79a018fa3688db938c8458e58dfe8e252d7bfc2d355e91101"} Jul 24 17:53:03.391170 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:03.391175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" event={"ID":"4821ddfb-f57a-4667-b34f-015b070b9bf3","Type":"ContainerStarted","Data":"ef45101bd6aabc8817a262f9c36a8193eb3fb59542f74a6925ce3be5a9fa1a4c"} Jul 24 17:53:03.410395 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:03.410322 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" podStartSLOduration=1.253794051 podStartE2EDuration="1.41030581s" totalImagesPullingTime="156.511759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:53:03.408609402 +0000 UTC m=+599.609841802" watchObservedRunningTime="2026-07-24 17:53:03.41030581 +0000 UTC m=+599.611538246" Jul 24 17:53:12.669097 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:12.669055 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:12.670428 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:12.669113 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:12.681946 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:12.681911 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:13.457212 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:13.457178 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx" Jul 24 17:53:35.905125 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:35.905036 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:53:35.905684 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:35.905280 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-86d6d5d896-hldh9" podUID="65946d8b-e317-4b86-a712-74fd9efcdaa6" containerName="authorino" containerID="cri-o://131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5" gracePeriod=30 Jul 24 17:53:36.159592 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.159516 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:53:36.229916 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.229880 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert\") pod \"65946d8b-e317-4b86-a712-74fd9efcdaa6\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " Jul 24 17:53:36.230106 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.229954 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7sfc\" (UniqueName: \"kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc\") pod \"65946d8b-e317-4b86-a712-74fd9efcdaa6\" (UID: \"65946d8b-e317-4b86-a712-74fd9efcdaa6\") " Jul 24 17:53:36.232090 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.232062 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc" (OuterVolumeSpecName: "kube-api-access-l7sfc") pod "65946d8b-e317-4b86-a712-74fd9efcdaa6" (UID: "65946d8b-e317-4b86-a712-74fd9efcdaa6"). InnerVolumeSpecName "kube-api-access-l7sfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 17:53:36.240637 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.240597 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "65946d8b-e317-4b86-a712-74fd9efcdaa6" (UID: "65946d8b-e317-4b86-a712-74fd9efcdaa6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 17:53:36.331314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.331277 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65946d8b-e317-4b86-a712-74fd9efcdaa6-tls-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:53:36.331314 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.331308 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7sfc\" (UniqueName: \"kubernetes.io/projected/65946d8b-e317-4b86-a712-74fd9efcdaa6-kube-api-access-l7sfc\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 17:53:36.576227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.576117 2566 generic.go:356] "Generic (PLEG): container finished" podID="65946d8b-e317-4b86-a712-74fd9efcdaa6" containerID="131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5" exitCode=0 Jul 24 17:53:36.576227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.576164 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86d6d5d896-hldh9" Jul 24 17:53:36.576227 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.576206 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86d6d5d896-hldh9" event={"ID":"65946d8b-e317-4b86-a712-74fd9efcdaa6","Type":"ContainerDied","Data":"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5"} Jul 24 17:53:36.576621 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.576263 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86d6d5d896-hldh9" event={"ID":"65946d8b-e317-4b86-a712-74fd9efcdaa6","Type":"ContainerDied","Data":"7fb11c282d18f5e0c8225357388e82bdff74fa38acab4ad0450399f7e241c5c2"} Jul 24 17:53:36.576621 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.576285 2566 scope.go:117] "RemoveContainer" containerID="131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5" Jul 24 17:53:36.585945 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.585924 2566 scope.go:117] "RemoveContainer" containerID="131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5" Jul 24 17:53:36.586274 ip-10-0-132-71 kubenswrapper[2566]: E0724 17:53:36.586248 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5\": container with ID starting with 131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5 not found: ID does not exist" containerID="131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5" Jul 24 17:53:36.586390 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.586285 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5"} err="failed to get container status \"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5\": rpc error: code = NotFound desc = could not find container \"131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5\": container with ID starting with 131beda6a3eb357d0e2f84f36bf94b80c9d8fda94aa02be5db3f1b598e419ac5 not found: ID does not exist" Jul 24 17:53:36.596116 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.596084 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:53:36.603918 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:36.603883 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86d6d5d896-hldh9"] Jul 24 17:53:38.248911 ip-10-0-132-71 kubenswrapper[2566]: I0724 17:53:38.248873 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65946d8b-e317-4b86-a712-74fd9efcdaa6" path="/var/lib/kubelet/pods/65946d8b-e317-4b86-a712-74fd9efcdaa6/volumes" Jul 24 18:00:00.174147 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.174108 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc"] Jul 24 18:00:00.175543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.174462 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65946d8b-e317-4b86-a712-74fd9efcdaa6" containerName="authorino" Jul 24 18:00:00.175543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.174474 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="65946d8b-e317-4b86-a712-74fd9efcdaa6" containerName="authorino" Jul 24 18:00:00.175543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.174561 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="65946d8b-e317-4b86-a712-74fd9efcdaa6" containerName="authorino" Jul 24 18:00:00.176214 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.176198 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" Jul 24 18:00:00.179484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.179463 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dgh7n\"" Jul 24 18:00:00.186008 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.185985 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" Jul 24 18:00:00.242487 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.242444 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc"] Jul 24 18:00:00.525548 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.525515 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc"] Jul 24 18:00:00.527432 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:00:00.527395 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8302a30a_b70f_49c0_b115_604dc9eec661.slice/crio-8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075 WatchSource:0}: Error finding container 8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075: Status 404 returned error can't find the container with id 8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075 Jul 24 18:00:00.529495 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.529473 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 18:00:00.570001 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:00.569967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" event={"ID":"8302a30a-b70f-49c0-b115-604dc9eec661","Type":"ContainerStarted","Data":"8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075"} Jul 24 18:00:03.592024 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:03.591943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" event={"ID":"8302a30a-b70f-49c0-b115-604dc9eec661","Type":"ContainerStarted","Data":"c719ff7d76b512b2a2b0896bed78d28c37d95422c52144c1fb6f495b16f9f3b1"} Jul 24 18:00:03.608495 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:03.608434 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" podStartSLOduration=1.211467356 podStartE2EDuration="3.608418541s" totalImagesPullingTime="2.396951185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:00:03.606443054 +0000 UTC m=+1019.807675444" watchObservedRunningTime="2026-07-24 18:00:03.608418541 +0000 UTC m=+1019.809650931" Jul 24 18:00:04.597976 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:04.597938 2566 generic.go:356] "Generic (PLEG): container finished" podID="8302a30a-b70f-49c0-b115-604dc9eec661" containerID="c719ff7d76b512b2a2b0896bed78d28c37d95422c52144c1fb6f495b16f9f3b1" exitCode=0 Jul 24 18:00:04.598378 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:04.598026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" event={"ID":"8302a30a-b70f-49c0-b115-604dc9eec661","Type":"ContainerDied","Data":"c719ff7d76b512b2a2b0896bed78d28c37d95422c52144c1fb6f495b16f9f3b1"} Jul 24 18:00:05.731530 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:05.731505 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" Jul 24 18:00:06.608335 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:06.608304 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" Jul 24 18:00:06.608511 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:06.608350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748600-m2ngc" event={"ID":"8302a30a-b70f-49c0-b115-604dc9eec661","Type":"ContainerDied","Data":"8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075"} Jul 24 18:00:06.608511 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:00:06.608407 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bddd4f01b643bc21c4ab52b368e81fedf388272bc2ee104928a51e7b1525075" Jul 24 18:04:02.682841 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.682762 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:02.683291 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.683141 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8302a30a-b70f-49c0-b115-604dc9eec661" containerName="cleanup" Jul 24 18:04:02.683291 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.683153 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8302a30a-b70f-49c0-b115-604dc9eec661" containerName="cleanup" Jul 24 18:04:02.683291 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.683220 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8302a30a-b70f-49c0-b115-604dc9eec661" containerName="cleanup" Jul 24 18:04:02.685247 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.685231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.688100 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.688078 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-20857293-kserve-self-signed-certs\"" Jul 24 18:04:02.712913 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.712884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:02.868192 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.868414 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.868414 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.868414 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.868585 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9bd\" (UniqueName: \"kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.868585 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.868479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970006 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.969886 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970006 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.969965 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970252 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9bd\" (UniqueName: \"kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970252 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970048 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970252 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970252 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970129 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970511 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970424 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970570 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970530 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.970609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.970567 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.972313 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.972295 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.972427 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.972410 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.978191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.978170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9bd\" (UniqueName: \"kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd\") pod \"e2e-same-id-a-20857293-kserve-86778f7cdc-5462r\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:02.995181 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:02.995143 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:03.130785 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:03.130758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:03.132434 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:04:03.132395 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08385465_7b1f_41f9_8d80_e240eb6a3db3.slice/crio-5c0502e9a661d9e17298b189db2cc92551029960733ca0b079b83f98772272a5 WatchSource:0}: Error finding container 5c0502e9a661d9e17298b189db2cc92551029960733ca0b079b83f98772272a5: Status 404 returned error can't find the container with id 5c0502e9a661d9e17298b189db2cc92551029960733ca0b079b83f98772272a5 Jul 24 18:04:03.864095 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:03.864047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" event={"ID":"08385465-7b1f-41f9-8d80-e240eb6a3db3","Type":"ContainerStarted","Data":"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf"} Jul 24 18:04:03.864095 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:03.864103 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" event={"ID":"08385465-7b1f-41f9-8d80-e240eb6a3db3","Type":"ContainerStarted","Data":"5c0502e9a661d9e17298b189db2cc92551029960733ca0b079b83f98772272a5"} Jul 24 18:04:03.886815 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:03.886757 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" podStartSLOduration=1.886739313 podStartE2EDuration="1.886739313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:04:03.884728211 +0000 UTC m=+1260.085960592" watchObservedRunningTime="2026-07-24 18:04:03.886739313 +0000 UTC m=+1260.087971729" Jul 24 18:04:12.996328 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:12.996223 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:12.996328 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:12.996303 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:13.008682 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:13.008648 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:13.929396 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:13.929351 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:30.445496 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:30.445460 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:45.464781 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.464721 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" podUID="08385465-7b1f-41f9-8d80-e240eb6a3db3" containerName="main" containerID="cri-o://848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf" gracePeriod=45 Jul 24 18:04:45.725272 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.725205 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:45.848339 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848306 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848542 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848399 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848542 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848434 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848542 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848466 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848542 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848506 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd9bd\" (UniqueName: \"kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848774 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs\") pod \"08385465-7b1f-41f9-8d80-e240eb6a3db3\" (UID: \"08385465-7b1f-41f9-8d80-e240eb6a3db3\") " Jul 24 18:04:45.848774 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848619 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home" (OuterVolumeSpecName: "home") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:04:45.848774 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848750 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache" (OuterVolumeSpecName: "model-cache") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:04:45.848908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848787 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:04:45.848908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848896 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-home\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:45.848981 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848914 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-model-cache\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:45.848981 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.848935 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-tmp-dir\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:45.850787 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.850752 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd" (OuterVolumeSpecName: "kube-api-access-pd9bd") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "kube-api-access-pd9bd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:04:45.850902 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.850803 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 18:04:45.850902 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.850801 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm" (OuterVolumeSpecName: "dshm") pod "08385465-7b1f-41f9-8d80-e240eb6a3db3" (UID: "08385465-7b1f-41f9-8d80-e240eb6a3db3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:04:45.949627 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.949584 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08385465-7b1f-41f9-8d80-e240eb6a3db3-dshm\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:45.949627 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.949621 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd9bd\" (UniqueName: \"kubernetes.io/projected/08385465-7b1f-41f9-8d80-e240eb6a3db3-kube-api-access-pd9bd\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:45.949627 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:45.949636 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08385465-7b1f-41f9-8d80-e240eb6a3db3-tls-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:04:46.083679 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.083583 2566 generic.go:356] "Generic (PLEG): container finished" podID="08385465-7b1f-41f9-8d80-e240eb6a3db3" containerID="848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf" exitCode=0 Jul 24 18:04:46.083679 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.083633 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" Jul 24 18:04:46.083679 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.083666 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" event={"ID":"08385465-7b1f-41f9-8d80-e240eb6a3db3","Type":"ContainerDied","Data":"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf"} Jul 24 18:04:46.083908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.083711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r" event={"ID":"08385465-7b1f-41f9-8d80-e240eb6a3db3","Type":"ContainerDied","Data":"5c0502e9a661d9e17298b189db2cc92551029960733ca0b079b83f98772272a5"} Jul 24 18:04:46.083908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.083731 2566 scope.go:117] "RemoveContainer" containerID="848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf" Jul 24 18:04:46.093095 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.093074 2566 scope.go:117] "RemoveContainer" containerID="848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf" Jul 24 18:04:46.093410 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:04:46.093384 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf\": container with ID starting with 848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf not found: ID does not exist" containerID="848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf" Jul 24 18:04:46.093513 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.093419 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf"} err="failed to get container status \"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf\": rpc error: code = NotFound desc = could not find container \"848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf\": container with ID starting with 848212368af087e071aaff325f1f811258c6b33fb4e60a472285cdf1cfc0d5cf not found: ID does not exist" Jul 24 18:04:46.105743 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.105710 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:46.110022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.109999 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-20857293-kserve-86778f7cdc-5462r"] Jul 24 18:04:46.249322 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:04:46.249281 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08385465-7b1f-41f9-8d80-e240eb6a3db3" path="/var/lib/kubelet/pods/08385465-7b1f-41f9-8d80-e240eb6a3db3/volumes" Jul 24 18:08:06.020460 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.020422 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:06.021069 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.021005 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08385465-7b1f-41f9-8d80-e240eb6a3db3" containerName="main" Jul 24 18:08:06.021069 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.021028 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="08385465-7b1f-41f9-8d80-e240eb6a3db3" containerName="main" Jul 24 18:08:06.021193 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.021127 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="08385465-7b1f-41f9-8d80-e240eb6a3db3" containerName="main" Jul 24 18:08:06.023249 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.023225 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.025792 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.025767 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-096012db-openshift-default-dockercfg-77m4g\"" Jul 24 18:08:06.026040 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.026023 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:08:06.044030 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.043999 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:06.048391 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048531 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048531 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048662 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048662 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048662 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048624 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r8sb\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048796 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048796 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.048796 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.048760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149456 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149456 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149456 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149456 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149666 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149666 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149666 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r8sb\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149827 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149732 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149889 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149848 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.149945 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149927 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.150003 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.150003 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.149976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.151659 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.151631 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.151911 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.151895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.158269 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.158246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.158424 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.158270 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r8sb\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb\") pod \"e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.334284 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.334185 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:06.476893 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.476864 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:06.478654 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:08:06.478625 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82422ea8_86b5_4a7b_9532_681aba246d6b.slice/crio-486f1aa3f9c346a6ce35182db80eea81b8a3f9d358d95b692fc3576ee2702063 WatchSource:0}: Error finding container 486f1aa3f9c346a6ce35182db80eea81b8a3f9d358d95b692fc3576ee2702063: Status 404 returned error can't find the container with id 486f1aa3f9c346a6ce35182db80eea81b8a3f9d358d95b692fc3576ee2702063 Jul 24 18:08:06.480972 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.480934 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:06.481084 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.481026 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:06.481084 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:06.481076 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:07.106437 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:07.106396 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerStarted","Data":"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262"} Jul 24 18:08:07.106870 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:07.106448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerStarted","Data":"486f1aa3f9c346a6ce35182db80eea81b8a3f9d358d95b692fc3576ee2702063"} Jul 24 18:08:07.128861 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:07.128801 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podStartSLOduration=2.128785085 podStartE2EDuration="2.128785085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:07.127445188 +0000 UTC m=+1503.328677578" watchObservedRunningTime="2026-07-24 18:08:07.128785085 +0000 UTC m=+1503.330017473" Jul 24 18:08:07.334880 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:07.334844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:07.340728 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:07.340701 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:08.112235 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:08.112207 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:08.113294 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:08.113272 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:22.187806 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.187718 2566 generic.go:356] "Generic (PLEG): container finished" podID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerID="f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262" exitCode=0 Jul 24 18:08:22.187806 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.187792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerDied","Data":"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262"} Jul 24 18:08:22.188256 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.188178 2566 scope.go:117] "RemoveContainer" containerID="f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262" Jul 24 18:08:22.189205 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.189178 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:22.189260 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.189246 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:22.189304 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:22.189277 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:23.113055 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:23.113024 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:23.198683 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:23.198643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerStarted","Data":"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555"} Jul 24 18:08:23.335108 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:23.335072 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:23.336313 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:23.336278 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:23.336476 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:23.336333 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:24.335594 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:24.335557 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:24.335959 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:24.335621 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:25.334750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:25.334712 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:25.334938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:25.334766 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:26.335540 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:26.335460 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:26.335540 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:26.335515 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:27.334823 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:27.334785 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:27.335012 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:27.334846 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:28.335640 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:28.335597 2566 patch_prober.go:28] interesting pod/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 24 18:08:28.336114 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:28.335663 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 24 18:08:29.338597 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:29.338563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:29.339084 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:29.338943 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:29.339709 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:29.339688 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:49.359752 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:49.359713 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:49.360332 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:49.360013 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" containerID="cri-o://e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555" gracePeriod=30 Jul 24 18:08:50.442732 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.442657 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:08:50.445266 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.445244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.448402 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.448384 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-35d23e85-gw-openshift-default-dockercfg-j4z9s\"" Jul 24 18:08:50.459457 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.459431 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:08:50.567643 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.567643 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567642 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.567903 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.567903 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.567903 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.567903 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27r8q\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.568122 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.568122 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.567958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.568122 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.568035 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669515 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669515 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669644 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27r8q\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669712 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.669778 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.670136 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669951 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.670136 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.669989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.670427 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.670398 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.670574 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.670457 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.670768 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.670750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.672012 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.671991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.672677 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.672656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.678338 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.678309 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.678749 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.678733 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27r8q\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q\") pod \"e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.759121 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.759027 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:50.907349 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.907316 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:08:50.908707 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:08:50.908670 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be68659_87c5_495a_bd43_1761b0e623c8.slice/crio-d239666e5c33f74c903fe2be5ea8b6fab8f1e9a7da8ae0cd730969895ae9492e WatchSource:0}: Error finding container d239666e5c33f74c903fe2be5ea8b6fab8f1e9a7da8ae0cd730969895ae9492e: Status 404 returned error can't find the container with id d239666e5c33f74c903fe2be5ea8b6fab8f1e9a7da8ae0cd730969895ae9492e Jul 24 18:08:50.911495 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.911460 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:50.911569 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.911543 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:50.911608 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:50.911587 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:08:51.346613 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:51.346576 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerStarted","Data":"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058"} Jul 24 18:08:51.346792 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:51.346632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerStarted","Data":"d239666e5c33f74c903fe2be5ea8b6fab8f1e9a7da8ae0cd730969895ae9492e"} Jul 24 18:08:51.370373 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:51.370310 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podStartSLOduration=1.370293093 podStartE2EDuration="1.370293093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:08:51.368001771 +0000 UTC m=+1547.569234197" watchObservedRunningTime="2026-07-24 18:08:51.370293093 +0000 UTC m=+1547.571525483" Jul 24 18:08:51.759861 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:51.759821 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:51.764683 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:51.764654 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:52.351275 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:52.351246 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:52.352425 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:52.352404 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:08:54.606848 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.606823 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:54.704876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704785 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.704876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704827 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.704876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704849 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.704876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704878 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704934 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.704962 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r8sb\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705032 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705054 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket\") pod \"82422ea8-86b5-4a7b-9532-681aba246d6b\" (UID: \"82422ea8-86b5-4a7b-9532-681aba246d6b\") " Jul 24 18:08:54.705199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705150 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705241 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705257 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data" (OuterVolumeSpecName: "istio-data") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705411 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705477 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705498 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705513 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.705617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.705597 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:08:54.707147 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.707112 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:08:54.707279 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.707230 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb" (OuterVolumeSpecName: "kube-api-access-9r8sb") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "kube-api-access-9r8sb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:08:54.707513 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.707481 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:08:54.707609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.707531 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token" (OuterVolumeSpecName: "istio-token") pod "82422ea8-86b5-4a7b-9532-681aba246d6b" (UID: "82422ea8-86b5-4a7b-9532-681aba246d6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:08:54.806498 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806468 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.806498 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806496 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82422ea8-86b5-4a7b-9532-681aba246d6b-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.806701 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806510 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r8sb\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-kube-api-access-9r8sb\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.806701 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806522 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.806701 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806533 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:54.806701 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:54.806543 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82422ea8-86b5-4a7b-9532-681aba246d6b-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:08:55.368413 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.368353 2566 generic.go:356] "Generic (PLEG): container finished" podID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerID="e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555" exitCode=0 Jul 24 18:08:55.368413 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.368418 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" Jul 24 18:08:55.368628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.368449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerDied","Data":"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555"} Jul 24 18:08:55.368628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.368503 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl" event={"ID":"82422ea8-86b5-4a7b-9532-681aba246d6b","Type":"ContainerDied","Data":"486f1aa3f9c346a6ce35182db80eea81b8a3f9d358d95b692fc3576ee2702063"} Jul 24 18:08:55.368628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.368526 2566 scope.go:117] "RemoveContainer" containerID="e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555" Jul 24 18:08:55.378704 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.378685 2566 scope.go:117] "RemoveContainer" containerID="f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262" Jul 24 18:08:55.386978 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.386961 2566 scope.go:117] "RemoveContainer" containerID="e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555" Jul 24 18:08:55.387239 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:08:55.387217 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555\": container with ID starting with e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555 not found: ID does not exist" containerID="e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555" Jul 24 18:08:55.387315 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.387255 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555"} err="failed to get container status \"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555\": rpc error: code = NotFound desc = could not find container \"e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555\": container with ID starting with e54e05cd5b2b9ece1ca12bc0401af8e1a7f49a7a21b2ffa3aa4fd0e4c518d555 not found: ID does not exist" Jul 24 18:08:55.387315 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.387279 2566 scope.go:117] "RemoveContainer" containerID="f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262" Jul 24 18:08:55.388033 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:08:55.388016 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262\": container with ID starting with f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262 not found: ID does not exist" containerID="f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262" Jul 24 18:08:55.388076 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.388038 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262"} err="failed to get container status \"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262\": rpc error: code = NotFound desc = could not find container \"f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262\": container with ID starting with f2392a52ad5d31eefc99e5f83ce2f82b8a4fda5971eba6e523459adb063bc262 not found: ID does not exist" Jul 24 18:08:55.392211 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.392187 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:55.395963 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:55.395941 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-096012db-openshift-default-c4845cdd8-mxgbl"] Jul 24 18:08:56.249388 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:08:56.249342 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" path="/var/lib/kubelet/pods/82422ea8-86b5-4a7b-9532-681aba246d6b/volumes" Jul 24 18:09:06.437145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.437112 2566 generic.go:356] "Generic (PLEG): container finished" podID="1be68659-87c5-495a-bd43-1761b0e623c8" containerID="174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058" exitCode=0 Jul 24 18:09:06.437555 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.437166 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerDied","Data":"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058"} Jul 24 18:09:06.437599 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.437554 2566 scope.go:117] "RemoveContainer" containerID="174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058" Jul 24 18:09:06.438543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.438509 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:06.438604 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.438583 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:06.438639 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:06.438613 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:09:07.352101 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:07.352066 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:07.444351 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:07.444316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerStarted","Data":"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8"} Jul 24 18:09:07.760159 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:07.760066 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:07.761238 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:07.761213 2566 patch_prober.go:28] interesting pod/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 18:09:07.761354 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:07.761257 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 18:09:08.759735 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:08.759697 2566 patch_prober.go:28] interesting pod/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 18:09:08.760233 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:08.759761 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 18:09:09.759676 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:09.759639 2566 patch_prober.go:28] interesting pod/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 18:09:09.759860 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:09.759694 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 18:09:10.759879 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:10.759840 2566 patch_prober.go:28] interesting pod/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 18:09:10.760330 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:10.759897 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 18:09:11.760108 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:11.760067 2566 patch_prober.go:28] interesting pod/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 24 18:09:11.760539 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:11.760123 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 24 18:09:12.763887 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:12.763854 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:12.764328 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:12.764147 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:12.764950 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:12.764928 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:17.857972 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:17.857933 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:09:17.858445 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:17.858187 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" containerID="cri-o://fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8" gracePeriod=30 Jul 24 18:09:22.302501 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302462 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:09:22.302916 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302829 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.302916 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302842 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.302916 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302857 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.302916 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302863 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.303056 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302928 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.303056 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.302937 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="82422ea8-86b5-4a7b-9532-681aba246d6b" containerName="istio-proxy" Jul 24 18:09:22.305188 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.305165 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.308148 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.308126 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f37130a8\"" Jul 24 18:09:22.308148 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.308146 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f37130a8-dockercfg-pkghr\"" Jul 24 18:09:22.318788 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.318759 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:09:22.470164 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.470122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.470397 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.470218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvrc\" (UniqueName: \"kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.571847 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.571757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvrc\" (UniqueName: \"kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.571997 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.571875 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.572590 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.572571 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.580572 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.580541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvrc\" (UniqueName: \"kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc\") pod \"payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.615915 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.615880 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:22.756022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:22.755995 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:09:22.758154 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:09:22.758128 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba86f70_6bc8_4442_87dc_60397dec91cb.slice/crio-394748a5393911a0697291339768284ddc093aa96d2955297d4d74d446c16f35 WatchSource:0}: Error finding container 394748a5393911a0697291339768284ddc093aa96d2955297d4d74d446c16f35: Status 404 returned error can't find the container with id 394748a5393911a0697291339768284ddc093aa96d2955297d4d74d446c16f35 Jul 24 18:09:23.096258 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.096234 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:23.278075 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.277976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278075 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278032 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278116 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278137 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27r8q\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278169 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278206 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278242 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278269 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278310 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo\") pod \"1be68659-87c5-495a-bd43-1761b0e623c8\" (UID: \"1be68659-87c5-495a-bd43-1761b0e623c8\") " Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278454 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278460 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278698 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1be68659-87c5-495a-bd43-1761b0e623c8-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278714 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278724 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.278751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278733 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:23.279028 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.278791 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:23.280878 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.280855 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:09:23.280964 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.280884 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:09:23.280964 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.280895 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:09:23.280964 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.280897 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q" (OuterVolumeSpecName: "kube-api-access-27r8q") pod "1be68659-87c5-495a-bd43-1761b0e623c8" (UID: "1be68659-87c5-495a-bd43-1761b0e623c8"). InnerVolumeSpecName "kube-api-access-27r8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:09:23.380097 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380060 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380097 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380099 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27r8q\" (UniqueName: \"kubernetes.io/projected/1be68659-87c5-495a-bd43-1761b0e623c8-kube-api-access-27r8q\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380117 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380132 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380145 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380158 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1be68659-87c5-495a-bd43-1761b0e623c8-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.380564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.380171 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1be68659-87c5-495a-bd43-1761b0e623c8-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:23.541237 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.541120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" event={"ID":"9ba86f70-6bc8-4442-87dc-60397dec91cb","Type":"ContainerStarted","Data":"e9d0740bd6a35c50e863b16a0a2d4af75f3a3d6157015059e43b29f8601fef92"} Jul 24 18:09:23.541237 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.541216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" event={"ID":"9ba86f70-6bc8-4442-87dc-60397dec91cb","Type":"ContainerStarted","Data":"394748a5393911a0697291339768284ddc093aa96d2955297d4d74d446c16f35"} Jul 24 18:09:23.541513 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.541248 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:23.542957 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.542934 2566 generic.go:356] "Generic (PLEG): container finished" podID="1be68659-87c5-495a-bd43-1761b0e623c8" containerID="fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8" exitCode=0 Jul 24 18:09:23.543071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.542977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerDied","Data":"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8"} Jul 24 18:09:23.543071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.543004 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" Jul 24 18:09:23.543071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.543017 2566 scope.go:117] "RemoveContainer" containerID="fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8" Jul 24 18:09:23.543224 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.543008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp" event={"ID":"1be68659-87c5-495a-bd43-1761b0e623c8","Type":"ContainerDied","Data":"d239666e5c33f74c903fe2be5ea8b6fab8f1e9a7da8ae0cd730969895ae9492e"} Jul 24 18:09:23.553235 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.553215 2566 scope.go:117] "RemoveContainer" containerID="174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058" Jul 24 18:09:23.560181 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.560129 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" podStartSLOduration=1.560114174 podStartE2EDuration="1.560114174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:09:23.558940489 +0000 UTC m=+1579.760172878" watchObservedRunningTime="2026-07-24 18:09:23.560114174 +0000 UTC m=+1579.761346563" Jul 24 18:09:23.562537 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.562516 2566 scope.go:117] "RemoveContainer" containerID="fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8" Jul 24 18:09:23.562829 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:09:23.562807 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8\": container with ID starting with fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8 not found: ID does not exist" containerID="fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8" Jul 24 18:09:23.562882 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.562838 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8"} err="failed to get container status \"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8\": rpc error: code = NotFound desc = could not find container \"fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8\": container with ID starting with fadf64f0a84b6f7c75c26e6d1c3529ee70e569dff964103acab429b95f0ff1e8 not found: ID does not exist" Jul 24 18:09:23.562882 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.562855 2566 scope.go:117] "RemoveContainer" containerID="174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058" Jul 24 18:09:23.563093 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:09:23.563075 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058\": container with ID starting with 174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058 not found: ID does not exist" containerID="174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058" Jul 24 18:09:23.563151 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.563096 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058"} err="failed to get container status \"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058\": rpc error: code = NotFound desc = could not find container \"174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058\": container with ID starting with 174b7f4d1e08f35af664c45a26d02fd03967a731df1ccce82736d7acd537c058 not found: ID does not exist" Jul 24 18:09:23.575407 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.575379 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:09:23.581519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:23.581491 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-35d23e85-gw-openshift-default-6c5cd68d88-7c5qp"] Jul 24 18:09:24.255026 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:24.254994 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" path="/var/lib/kubelet/pods/1be68659-87c5-495a-bd43-1761b0e623c8/volumes" Jul 24 18:09:34.551732 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:34.551698 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:47.091961 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.091915 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls"] Jul 24 18:09:47.092505 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092485 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.092593 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092507 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.092593 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092521 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.092593 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092529 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.092756 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092642 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.092756 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.092658 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1be68659-87c5-495a-bd43-1761b0e623c8" containerName="istio-proxy" Jul 24 18:09:47.094910 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.094882 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:47.097875 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.097854 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dgh7n\"" Jul 24 18:09:47.097988 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.097923 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 18:09:47.104490 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.104462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls"] Jul 24 18:09:47.196956 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.196919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f37130a8-dbwls\" (UID: \"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:47.297956 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.297913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f37130a8-dbwls\" (UID: \"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:47.298591 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.298572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f37130a8-dbwls\" (UID: \"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:47.407800 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.407760 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:47.541246 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.541220 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls"] Jul 24 18:09:47.543059 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:09:47.543030 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47ac4cc_ebc5_4894_9fc1_bb146f4ec00a.slice/crio-16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02 WatchSource:0}: Error finding container 16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02: Status 404 returned error can't find the container with id 16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02 Jul 24 18:09:47.674645 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:47.674558 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" event={"ID":"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a","Type":"ContainerStarted","Data":"16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02"} Jul 24 18:09:48.681258 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:48.681170 2566 generic.go:356] "Generic (PLEG): container finished" podID="f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" containerID="b2e3bd63461fb36c216b8fb5232bbfb2f28c27b6f529a72bd53434b9a259833f" exitCode=0 Jul 24 18:09:48.681641 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:48.681257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" event={"ID":"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a","Type":"ContainerDied","Data":"b2e3bd63461fb36c216b8fb5232bbfb2f28c27b6f529a72bd53434b9a259833f"} Jul 24 18:09:49.828344 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:49.828320 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:49.922944 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:49.922913 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca\") pod \"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a\" (UID: \"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a\") " Jul 24 18:09:49.923246 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:49.923223 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" (UID: "f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:09:50.024327 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:50.024236 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a-maas-api-service-ca\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:50.692173 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:50.692138 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" Jul 24 18:09:50.692405 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:50.692133 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls" event={"ID":"f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a","Type":"ContainerDied","Data":"16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02"} Jul 24 18:09:50.692405 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:50.692276 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16804b6fe0855d92b7f74613d2f0250feb0d48214eba00817cb4cb1a286e0a02" Jul 24 18:09:57.603700 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:57.603612 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:09:57.604165 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:57.603872 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" podUID="9ba86f70-6bc8-4442-87dc-60397dec91cb" containerName="payload-processing" containerID="cri-o://e9d0740bd6a35c50e863b16a0a2d4af75f3a3d6157015059e43b29f8601fef92" gracePeriod=30 Jul 24 18:09:57.731237 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:57.731205 2566 generic.go:356] "Generic (PLEG): container finished" podID="9ba86f70-6bc8-4442-87dc-60397dec91cb" containerID="e9d0740bd6a35c50e863b16a0a2d4af75f3a3d6157015059e43b29f8601fef92" exitCode=0 Jul 24 18:09:57.731417 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:57.731283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" event={"ID":"9ba86f70-6bc8-4442-87dc-60397dec91cb","Type":"ContainerDied","Data":"e9d0740bd6a35c50e863b16a0a2d4af75f3a3d6157015059e43b29f8601fef92"} Jul 24 18:09:57.861492 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:57.861431 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:58.001192 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.001138 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xvrc\" (UniqueName: \"kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc\") pod \"9ba86f70-6bc8-4442-87dc-60397dec91cb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " Jul 24 18:09:58.001192 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.001193 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume\") pod \"9ba86f70-6bc8-4442-87dc-60397dec91cb\" (UID: \"9ba86f70-6bc8-4442-87dc-60397dec91cb\") " Jul 24 18:09:58.001678 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.001652 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ba86f70-6bc8-4442-87dc-60397dec91cb" (UID: "9ba86f70-6bc8-4442-87dc-60397dec91cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:09:58.003278 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.003254 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc" (OuterVolumeSpecName: "kube-api-access-8xvrc") pod "9ba86f70-6bc8-4442-87dc-60397dec91cb" (UID: "9ba86f70-6bc8-4442-87dc-60397dec91cb"). InnerVolumeSpecName "kube-api-access-8xvrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:09:58.102041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.101987 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xvrc\" (UniqueName: \"kubernetes.io/projected/9ba86f70-6bc8-4442-87dc-60397dec91cb-kube-api-access-8xvrc\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:58.102041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.102035 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ba86f70-6bc8-4442-87dc-60397dec91cb-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:09:58.738094 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.738057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" event={"ID":"9ba86f70-6bc8-4442-87dc-60397dec91cb","Type":"ContainerDied","Data":"394748a5393911a0697291339768284ddc093aa96d2955297d4d74d446c16f35"} Jul 24 18:09:58.738596 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.738114 2566 scope.go:117] "RemoveContainer" containerID="e9d0740bd6a35c50e863b16a0a2d4af75f3a3d6157015059e43b29f8601fef92" Jul 24 18:09:58.738596 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.738072 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb" Jul 24 18:09:58.760876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.760846 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:09:58.767342 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:09:58.767311 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f37130a8-85454675cd-9jdfb"] Jul 24 18:10:00.249117 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:00.249083 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba86f70-6bc8-4442-87dc-60397dec91cb" path="/var/lib/kubelet/pods/9ba86f70-6bc8-4442-87dc-60397dec91cb/volumes" Jul 24 18:10:02.184737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:02.184695 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls"] Jul 24 18:10:02.186767 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:02.186746 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f37130a8-dbwls"] Jul 24 18:10:02.249372 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:02.249333 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" path="/var/lib/kubelet/pods/f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a/volumes" Jul 24 18:10:04.279015 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.278972 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:04.279429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279326 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" containerName="revoke-keys" Jul 24 18:10:04.279429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279336 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" containerName="revoke-keys" Jul 24 18:10:04.279429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279354 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba86f70-6bc8-4442-87dc-60397dec91cb" containerName="payload-processing" Jul 24 18:10:04.279429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279378 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba86f70-6bc8-4442-87dc-60397dec91cb" containerName="payload-processing" Jul 24 18:10:04.279587 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279453 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba86f70-6bc8-4442-87dc-60397dec91cb" containerName="payload-processing" Jul 24 18:10:04.279587 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.279464 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47ac4cc-ebc5-4894-9fc1-bb146f4ec00a" containerName="revoke-keys" Jul 24 18:10:04.283385 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.283369 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.285963 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.285938 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-294b6e13-openshift-default-dockercfg-5whzj\"" Jul 24 18:10:04.286454 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.286431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:10:04.296236 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.296201 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:04.360710 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360710 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.360980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360921 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9fx\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.361245 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.360989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.361245 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.361009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462324 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462324 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462329 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9fx\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.462908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.463022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462957 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.463022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.462990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.463262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.463236 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.463446 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.463428 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.464901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.464879 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.465120 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.465104 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.471230 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.471200 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.471524 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.471500 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9fx\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx\") pod \"e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.595746 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.595645 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:04.741968 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.741940 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:04.744058 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:10:04.744031 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcf7708_fd0b_4ea7_bf98_d1960e4b0a34.slice/crio-9c476719c4925df8f4dcc4bc11631393de71ed17aa3af0f6c9607b31468d444f WatchSource:0}: Error finding container 9c476719c4925df8f4dcc4bc11631393de71ed17aa3af0f6c9607b31468d444f: Status 404 returned error can't find the container with id 9c476719c4925df8f4dcc4bc11631393de71ed17aa3af0f6c9607b31468d444f Jul 24 18:10:04.746173 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.746137 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:04.746250 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.746206 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:04.746250 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.746235 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:04.772125 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:04.772090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerStarted","Data":"9c476719c4925df8f4dcc4bc11631393de71ed17aa3af0f6c9607b31468d444f"} Jul 24 18:10:05.779593 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:05.779551 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerStarted","Data":"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623"} Jul 24 18:10:05.810996 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:05.810938 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podStartSLOduration=1.810922099 podStartE2EDuration="1.810922099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:10:05.808853102 +0000 UTC m=+1622.010085517" watchObservedRunningTime="2026-07-24 18:10:05.810922099 +0000 UTC m=+1622.012154487" Jul 24 18:10:06.596748 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:06.596709 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:06.601692 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:06.601662 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:06.785057 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:06.785028 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:06.786019 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:06.786001 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:20.862754 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.862722 2566 generic.go:356] "Generic (PLEG): container finished" podID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerID="e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623" exitCode=0 Jul 24 18:10:20.863162 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.862792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerDied","Data":"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623"} Jul 24 18:10:20.863279 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.863259 2566 scope.go:117] "RemoveContainer" containerID="e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623" Jul 24 18:10:20.864283 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.864253 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:20.864380 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.864342 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:20.864429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:20.864414 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:10:21.785382 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:21.785321 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:21.870105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:21.870067 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerStarted","Data":"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b"} Jul 24 18:10:22.596582 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:22.596494 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:22.598223 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:22.598194 2566 patch_prober.go:28] interesting pod/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 18:10:22.598484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:22.598456 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 18:10:23.597334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:23.596846 2566 patch_prober.go:28] interesting pod/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 18:10:23.597334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:23.596928 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 18:10:24.596676 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:24.596635 2566 patch_prober.go:28] interesting pod/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 18:10:24.596987 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:24.596955 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 18:10:25.596245 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:25.596206 2566 patch_prober.go:28] interesting pod/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 18:10:25.596876 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:25.596844 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 18:10:26.596847 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:26.596805 2566 patch_prober.go:28] interesting pod/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 24 18:10:26.597477 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:26.596874 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 24 18:10:27.601262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:27.601228 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:27.601661 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:27.601579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:27.602339 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:27.602320 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:47.310124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:47.310086 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:47.310704 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:47.310347 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" containerID="cri-o://590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b" gracePeriod=30 Jul 24 18:10:52.548587 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.548560 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:52.719791 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719757 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719837 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719872 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719891 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719917 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719952 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.719980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720001 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p9fx\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720047 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert\") pod \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\" (UID: \"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34\") " Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720156 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720174 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720277 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720355 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720401 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720353 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data" (OuterVolumeSpecName: "istio-data") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:52.720556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.720517 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:10:52.722165 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.722136 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:10:52.722285 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.722181 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:10:52.722285 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.722253 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx" (OuterVolumeSpecName: "kube-api-access-4p9fx") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "kube-api-access-4p9fx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:10:52.722392 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.722325 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token" (OuterVolumeSpecName: "istio-token") pod "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" (UID: "afcf7708-fd0b-4ea7-bf98-d1960e4b0a34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:10:52.821642 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821601 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821642 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821637 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821642 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821651 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821899 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821662 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821899 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821674 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p9fx\" (UniqueName: \"kubernetes.io/projected/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-kube-api-access-4p9fx\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821899 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821685 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:52.821899 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:52.821697 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:10:53.049572 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.049483 2566 generic.go:356] "Generic (PLEG): container finished" podID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerID="590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b" exitCode=0 Jul 24 18:10:53.049572 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.049534 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" Jul 24 18:10:53.049788 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.049572 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerDied","Data":"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b"} Jul 24 18:10:53.049788 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.049630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx" event={"ID":"afcf7708-fd0b-4ea7-bf98-d1960e4b0a34","Type":"ContainerDied","Data":"9c476719c4925df8f4dcc4bc11631393de71ed17aa3af0f6c9607b31468d444f"} Jul 24 18:10:53.049788 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.049655 2566 scope.go:117] "RemoveContainer" containerID="590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b" Jul 24 18:10:53.062479 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.062453 2566 scope.go:117] "RemoveContainer" containerID="e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623" Jul 24 18:10:53.071177 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.071157 2566 scope.go:117] "RemoveContainer" containerID="590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b" Jul 24 18:10:53.071441 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:10:53.071423 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b\": container with ID starting with 590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b not found: ID does not exist" containerID="590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b" Jul 24 18:10:53.071489 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.071450 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b"} err="failed to get container status \"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b\": rpc error: code = NotFound desc = could not find container \"590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b\": container with ID starting with 590bcb86de3afefe386eb8c62e1a46817fa023edeca06d0ca80c71408d9c606b not found: ID does not exist" Jul 24 18:10:53.071489 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.071468 2566 scope.go:117] "RemoveContainer" containerID="e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623" Jul 24 18:10:53.071700 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:10:53.071677 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623\": container with ID starting with e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623 not found: ID does not exist" containerID="e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623" Jul 24 18:10:53.071745 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.071708 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623"} err="failed to get container status \"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623\": rpc error: code = NotFound desc = could not find container \"e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623\": container with ID starting with e25a743c20cb58ada1a884a8c052d49f01ee0e437e75fcc0a152499786c35623 not found: ID does not exist" Jul 24 18:10:53.077310 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.077285 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:53.079477 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:53.079459 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-294b6e13-openshift-default-59b44d8b9b-m9vfx"] Jul 24 18:10:54.250602 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:10:54.250570 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" path="/var/lib/kubelet/pods/afcf7708-fd0b-4ea7-bf98-d1960e4b0a34/volumes" Jul 24 18:12:27.188421 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.188385 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:12:27.189038 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.188755 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:12:27.189038 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.188767 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:12:27.189038 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.188840 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:12:27.189038 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.188847 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:12:27.191170 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.191151 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.195455 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.195433 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:12:27.195595 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.195550 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-72f7ddbd-openshift-default-dockercfg-mdqsx\"" Jul 24 18:12:27.219387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.219341 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:12:27.230690 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.230846 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.230846 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230823 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.230953 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.230953 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7zmp\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.231029 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.231029 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.230987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.231029 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.231024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.231142 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.231071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332346 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332346 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7zmp\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332474 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.332650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333021 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.332861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333021 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.333007 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333104 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.333066 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333216 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.333198 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333271 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.333208 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.333417 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.333392 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.335084 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.335065 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.335231 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.335211 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.341719 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.341693 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7zmp\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.341837 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.341719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token\") pod \"e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.502006 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.501914 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:27.653957 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.653928 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:12:27.655681 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:12:27.655653 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b70e3f9_da1a_4ebc_9044_d6983396c8e2.slice/crio-fb4fe638d45761558407d545a8196b625ce625e024309f7bd9002d76e76cb6c9 WatchSource:0}: Error finding container fb4fe638d45761558407d545a8196b625ce625e024309f7bd9002d76e76cb6c9: Status 404 returned error can't find the container with id fb4fe638d45761558407d545a8196b625ce625e024309f7bd9002d76e76cb6c9 Jul 24 18:12:27.657711 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.657680 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:27.657796 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.657748 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:27.657796 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:27.657774 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:28.554351 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:28.554310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerStarted","Data":"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c"} Jul 24 18:12:28.554754 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:28.554388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerStarted","Data":"fb4fe638d45761558407d545a8196b625ce625e024309f7bd9002d76e76cb6c9"} Jul 24 18:12:28.577829 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:28.577769 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podStartSLOduration=1.5777528109999999 podStartE2EDuration="1.577752811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:12:28.576096663 +0000 UTC m=+1764.777329051" watchObservedRunningTime="2026-07-24 18:12:28.577752811 +0000 UTC m=+1764.778985234" Jul 24 18:12:29.502587 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:29.502545 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:29.507577 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:29.507550 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:29.559210 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:29.559179 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:29.560187 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:29.560166 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:44.561170 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.561092 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:44.561170 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.561155 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:44.642492 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.642457 2566 generic.go:356] "Generic (PLEG): container finished" podID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerID="6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c" exitCode=0 Jul 24 18:12:44.642697 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.642537 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerDied","Data":"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c"} Jul 24 18:12:44.642980 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.642963 2566 scope.go:117] "RemoveContainer" containerID="6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c" Jul 24 18:12:44.643978 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.643947 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:44.644092 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.644027 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:44.644092 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:44.644073 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:12:45.649521 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:45.649482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerStarted","Data":"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79"} Jul 24 18:12:46.502568 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:46.502514 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:46.502807 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:46.502781 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:46.502884 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:46.502842 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:47.503024 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:47.502980 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:47.503545 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:47.503044 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:48.502648 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:48.502600 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:48.502836 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:48.502675 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:49.502644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:49.502603 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:49.503046 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:49.502667 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:50.503272 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:50.503228 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 18:12:50.503690 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:50.503310 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 18:12:51.506877 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:51.506846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:51.507276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:51.507104 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:12:51.507759 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:12:51.507736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:13:04.294931 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:04.294892 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:13:04.295398 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:04.295165 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" containerID="cri-o://71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79" gracePeriod=30 Jul 24 18:13:06.509098 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:06.509063 2566 patch_prober.go:28] interesting pod/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:13:06.509484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:06.509130 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:13:09.545900 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.545872 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:13:09.636262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636231 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636291 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7zmp\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636310 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636346 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636402 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636419 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636442 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636480 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636888 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636527 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token\") pod \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\" (UID: \"3b70e3f9-da1a-4ebc-9044-d6983396c8e2\") " Jul 24 18:13:09.636888 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636684 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:13:09.636888 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636759 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:13:09.636888 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636718 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:13:09.637102 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636898 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.637102 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636912 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:13:09.637102 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636917 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.637102 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.636967 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.637321 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.637113 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:13:09.638759 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.638729 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp" (OuterVolumeSpecName: "kube-api-access-t7zmp") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "kube-api-access-t7zmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:13:09.638759 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.638726 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:13:09.638983 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.638956 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:13:09.639093 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.639011 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b70e3f9-da1a-4ebc-9044-d6983396c8e2" (UID: "3b70e3f9-da1a-4ebc-9044-d6983396c8e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:13:09.738311 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738264 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.738311 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738300 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.738311 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738311 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.738311 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738319 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7zmp\" (UniqueName: \"kubernetes.io/projected/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-kube-api-access-t7zmp\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.738629 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738329 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.738629 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.738337 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b70e3f9-da1a-4ebc-9044-d6983396c8e2-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:13:09.780661 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.780623 2566 generic.go:356] "Generic (PLEG): container finished" podID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerID="71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79" exitCode=0 Jul 24 18:13:09.780804 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.780674 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" Jul 24 18:13:09.780804 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.780689 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerDied","Data":"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79"} Jul 24 18:13:09.780804 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.780778 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd" event={"ID":"3b70e3f9-da1a-4ebc-9044-d6983396c8e2","Type":"ContainerDied","Data":"fb4fe638d45761558407d545a8196b625ce625e024309f7bd9002d76e76cb6c9"} Jul 24 18:13:09.780924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.780809 2566 scope.go:117] "RemoveContainer" containerID="71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79" Jul 24 18:13:09.790341 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.790319 2566 scope.go:117] "RemoveContainer" containerID="6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c" Jul 24 18:13:09.798295 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.798277 2566 scope.go:117] "RemoveContainer" containerID="71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79" Jul 24 18:13:09.798567 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:13:09.798546 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79\": container with ID starting with 71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79 not found: ID does not exist" containerID="71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79" Jul 24 18:13:09.798630 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.798577 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79"} err="failed to get container status \"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79\": rpc error: code = NotFound desc = could not find container \"71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79\": container with ID starting with 71a306cf87f8432e03f4912acd339863849fdaafd761f1c3725851f34bf3ff79 not found: ID does not exist" Jul 24 18:13:09.798630 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.798596 2566 scope.go:117] "RemoveContainer" containerID="6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c" Jul 24 18:13:09.798818 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:13:09.798799 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c\": container with ID starting with 6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c not found: ID does not exist" containerID="6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c" Jul 24 18:13:09.798870 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.798823 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c"} err="failed to get container status \"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c\": rpc error: code = NotFound desc = could not find container \"6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c\": container with ID starting with 6482c2bbb5f2ee5ad8c8a301c7d63e4a74318fa0d0c604f90e1e1fa1a4a5308c not found: ID does not exist" Jul 24 18:13:09.805713 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.805690 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:13:09.809596 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:09.809574 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-72f7ddbd-openshift-default-765579b664-shrbd"] Jul 24 18:13:10.250191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:10.250158 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" path="/var/lib/kubelet/pods/3b70e3f9-da1a-4ebc-9044-d6983396c8e2/volumes" Jul 24 18:13:24.455329 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455292 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455668 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455680 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcf7708-fd0b-4ea7-bf98-d1960e4b0a34" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455690 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455696 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455712 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455718 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:24.455789 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.455771 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:24.458486 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.458469 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.462869 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.462846 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:13:24.463014 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.462887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-159379b5-openshift-default-dockercfg-548tw\"" Jul 24 18:13:24.481671 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.481644 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:13:24.575666 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.575845 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.575845 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.575845 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.575965 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.575965 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.575937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsh9\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.576144 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.576004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.576198 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.576172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.576198 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.576193 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677195 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677395 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677395 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677498 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677557 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677557 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677521 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677557 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677711 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677628 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677711 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677694 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677827 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677885 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677828 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsh9\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.677943 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.677887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.678130 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.678110 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.678497 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.678473 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.679824 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.679799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.679950 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.679805 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.685994 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.685974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.686299 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.686275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsh9\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9\") pod \"e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.770724 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.770642 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:24.908571 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.908544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:13:24.910561 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:13:24.910527 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a28f20_ed73_4472_8577_12c3e3b26100.slice/crio-a1ae4d0ca345dd028cd712e0ed96bac9816aa9686473434c7d9aae51d677edbc WatchSource:0}: Error finding container a1ae4d0ca345dd028cd712e0ed96bac9816aa9686473434c7d9aae51d677edbc: Status 404 returned error can't find the container with id a1ae4d0ca345dd028cd712e0ed96bac9816aa9686473434c7d9aae51d677edbc Jul 24 18:13:24.913124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.913086 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:24.913212 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.913157 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:24.913212 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:24.913186 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:25.871855 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:25.871819 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerStarted","Data":"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222"} Jul 24 18:13:25.872336 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:25.871868 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerStarted","Data":"a1ae4d0ca345dd028cd712e0ed96bac9816aa9686473434c7d9aae51d677edbc"} Jul 24 18:13:25.893940 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:25.893885 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podStartSLOduration=1.893868797 podStartE2EDuration="1.893868797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:13:25.892736637 +0000 UTC m=+1822.093969025" watchObservedRunningTime="2026-07-24 18:13:25.893868797 +0000 UTC m=+1822.095101208" Jul 24 18:13:26.771159 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:26.771120 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:26.776207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:26.776181 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:26.878423 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:26.878388 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:26.879334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:26.879310 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:28.678663 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.678632 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:13:28.679067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.679053 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b70e3f9-da1a-4ebc-9044-d6983396c8e2" containerName="istio-proxy" Jul 24 18:13:28.681290 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.681266 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.683940 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.683919 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-159379b5\"" Jul 24 18:13:28.692943 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.692913 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:13:28.716124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.716079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.716343 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.716327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24bnn\" (UniqueName: \"kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.817129 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.817079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.817129 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.817134 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24bnn\" (UniqueName: \"kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.817907 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.817885 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.828208 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.828163 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24bnn\" (UniqueName: \"kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn\") pod \"payload-processing-e2e-mt-159379b5-7c65c78574-fgx74\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:28.992554 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:28.992456 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:29.129463 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:29.129428 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:13:29.132632 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:13:29.132602 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844cd73f_63a8_4f3b_a7a0_046a7fa1e141.slice/crio-73244633e58124ee9b7aa0cb339fc598b3795665efee838b4831ef64e140bd88 WatchSource:0}: Error finding container 73244633e58124ee9b7aa0cb339fc598b3795665efee838b4831ef64e140bd88: Status 404 returned error can't find the container with id 73244633e58124ee9b7aa0cb339fc598b3795665efee838b4831ef64e140bd88 Jul 24 18:13:29.896150 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:29.896118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" event={"ID":"844cd73f-63a8-4f3b-a7a0-046a7fa1e141","Type":"ContainerStarted","Data":"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889"} Jul 24 18:13:29.896644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:29.896168 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" event={"ID":"844cd73f-63a8-4f3b-a7a0-046a7fa1e141","Type":"ContainerStarted","Data":"73244633e58124ee9b7aa0cb339fc598b3795665efee838b4831ef64e140bd88"} Jul 24 18:13:29.896644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:29.896244 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:29.916465 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:29.916412 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" podStartSLOduration=1.9163929149999999 podStartE2EDuration="1.916392915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:13:29.913593316 +0000 UTC m=+1826.114825915" watchObservedRunningTime="2026-07-24 18:13:29.916392915 +0000 UTC m=+1826.117625304" Jul 24 18:13:40.904060 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:40.904026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:13:41.880353 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.880315 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:41.880578 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.880411 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:41.964138 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.964054 2566 generic.go:356] "Generic (PLEG): container finished" podID="39a28f20-ed73-4472-8577-12c3e3b26100" containerID="48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222" exitCode=0 Jul 24 18:13:41.964138 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.964105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerDied","Data":"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222"} Jul 24 18:13:41.964575 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.964540 2566 scope.go:117] "RemoveContainer" containerID="48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222" Jul 24 18:13:41.965602 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.965569 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:41.965713 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.965643 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:41.965713 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:41.965679 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:13:42.971440 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:42.971406 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerStarted","Data":"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565"} Jul 24 18:13:43.770953 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:43.770919 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:43.771128 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:43.771103 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:43.771183 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:43.771163 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:44.771323 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:44.771287 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:44.771761 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:44.771345 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:45.771332 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:45.771294 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:45.771945 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:45.771355 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:46.772080 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:46.772041 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:46.772506 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:46.772102 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:47.772026 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:47.771980 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 24 18:13:47.772193 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:47.772056 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 24 18:13:48.775280 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:48.775252 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:48.775748 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:48.775563 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:13:48.776086 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:13:48.776069 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:14:17.887840 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:17.887804 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:14:17.888337 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:17.888021 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" podUID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" containerName="payload-processing" containerID="cri-o://61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889" gracePeriod=30 Jul 24 18:14:18.147629 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.147564 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:14:18.179824 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.179775 2566 generic.go:356] "Generic (PLEG): container finished" podID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" containerID="61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889" exitCode=0 Jul 24 18:14:18.180041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.179830 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" Jul 24 18:14:18.180041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.179845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" event={"ID":"844cd73f-63a8-4f3b-a7a0-046a7fa1e141","Type":"ContainerDied","Data":"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889"} Jul 24 18:14:18.180041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.179910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74" event={"ID":"844cd73f-63a8-4f3b-a7a0-046a7fa1e141","Type":"ContainerDied","Data":"73244633e58124ee9b7aa0cb339fc598b3795665efee838b4831ef64e140bd88"} Jul 24 18:14:18.180041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.179943 2566 scope.go:117] "RemoveContainer" containerID="61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889" Jul 24 18:14:18.182553 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.182527 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume\") pod \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " Jul 24 18:14:18.182681 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.182560 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24bnn\" (UniqueName: \"kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn\") pod \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\" (UID: \"844cd73f-63a8-4f3b-a7a0-046a7fa1e141\") " Jul 24 18:14:18.182919 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.182896 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "844cd73f-63a8-4f3b-a7a0-046a7fa1e141" (UID: "844cd73f-63a8-4f3b-a7a0-046a7fa1e141"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:14:18.184695 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.184670 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn" (OuterVolumeSpecName: "kube-api-access-24bnn") pod "844cd73f-63a8-4f3b-a7a0-046a7fa1e141" (UID: "844cd73f-63a8-4f3b-a7a0-046a7fa1e141"). InnerVolumeSpecName "kube-api-access-24bnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:14:18.196548 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.196523 2566 scope.go:117] "RemoveContainer" containerID="61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889" Jul 24 18:14:18.196876 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:14:18.196853 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889\": container with ID starting with 61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889 not found: ID does not exist" containerID="61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889" Jul 24 18:14:18.196943 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.196885 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889"} err="failed to get container status \"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889\": rpc error: code = NotFound desc = could not find container \"61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889\": container with ID starting with 61d5d07b5d8f7c1be38d8d35a05ddb1daec6bf4b62bc495b720d5568d8e36889 not found: ID does not exist" Jul 24 18:14:18.283657 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.283620 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24bnn\" (UniqueName: \"kubernetes.io/projected/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-kube-api-access-24bnn\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:14:18.283657 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.283648 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/844cd73f-63a8-4f3b-a7a0-046a7fa1e141-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:14:18.496286 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.496188 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:14:18.500308 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:18.500277 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-159379b5-7c65c78574-fgx74"] Jul 24 18:14:20.248850 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:14:20.248820 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" path="/var/lib/kubelet/pods/844cd73f-63a8-4f3b-a7a0-046a7fa1e141/volumes" Jul 24 18:15:14.071863 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:14.071827 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:15:14.072354 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:14.072146 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" containerID="cri-o://bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565" gracePeriod=30 Jul 24 18:15:18.777726 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:18.777687 2566 patch_prober.go:28] interesting pod/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:15:18.778099 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:18.777749 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:15:19.330735 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.330667 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:15:19.432323 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432282 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432534 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432534 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432449 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432534 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432488 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432534 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432517 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432562 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngsh9\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432588 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432616 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432648 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket\") pod \"39a28f20-ed73-4472-8577-12c3e3b26100\" (UID: \"39a28f20-ed73-4472-8577-12c3e3b26100\") " Jul 24 18:15:19.432737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432652 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:15:19.433006 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432931 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:15:19.433073 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.433018 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:15:19.433073 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.433045 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.433073 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.433064 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.433073 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.432699 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data" (OuterVolumeSpecName: "istio-data") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:15:19.433282 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.433155 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:15:19.434897 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.434870 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9" (OuterVolumeSpecName: "kube-api-access-ngsh9") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "kube-api-access-ngsh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:15:19.434897 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.434885 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token" (OuterVolumeSpecName: "istio-token") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:15:19.435093 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.435052 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:15:19.435093 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.435073 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39a28f20-ed73-4472-8577-12c3e3b26100" (UID: "39a28f20-ed73-4472-8577-12c3e3b26100"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:15:19.512128 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.512094 2566 generic.go:356] "Generic (PLEG): container finished" podID="39a28f20-ed73-4472-8577-12c3e3b26100" containerID="bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565" exitCode=0 Jul 24 18:15:19.512312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.512156 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" Jul 24 18:15:19.512312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.512187 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerDied","Data":"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565"} Jul 24 18:15:19.512312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.512269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8" event={"ID":"39a28f20-ed73-4472-8577-12c3e3b26100","Type":"ContainerDied","Data":"a1ae4d0ca345dd028cd712e0ed96bac9816aa9686473434c7d9aae51d677edbc"} Jul 24 18:15:19.512312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.512294 2566 scope.go:117] "RemoveContainer" containerID="bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565" Jul 24 18:15:19.521872 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.521856 2566 scope.go:117] "RemoveContainer" containerID="48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222" Jul 24 18:15:19.529974 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.529954 2566 scope.go:117] "RemoveContainer" containerID="bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565" Jul 24 18:15:19.530259 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:15:19.530235 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565\": container with ID starting with bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565 not found: ID does not exist" containerID="bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565" Jul 24 18:15:19.530382 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.530268 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565"} err="failed to get container status \"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565\": rpc error: code = NotFound desc = could not find container \"bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565\": container with ID starting with bc8cdd2571828a99f24d39a8830e32ef1a3c67ed33e3c7bb26caf7764184c565 not found: ID does not exist" Jul 24 18:15:19.530382 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.530288 2566 scope.go:117] "RemoveContainer" containerID="48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222" Jul 24 18:15:19.530563 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:15:19.530542 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222\": container with ID starting with 48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222 not found: ID does not exist" containerID="48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222" Jul 24 18:15:19.530599 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.530569 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222"} err="failed to get container status \"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222\": rpc error: code = NotFound desc = could not find container \"48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222\": container with ID starting with 48ac2d03ef5ec0cd6a1684b4ee59e0ffd0d250b00da328a77541470320fcd222 not found: ID does not exist" Jul 24 18:15:19.533761 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533744 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngsh9\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-kube-api-access-ngsh9\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533762 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39a28f20-ed73-4472-8577-12c3e3b26100-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533810 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533820 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533828 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39a28f20-ed73-4472-8577-12c3e3b26100-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533836 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39a28f20-ed73-4472-8577-12c3e3b26100-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.533842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.533843 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39a28f20-ed73-4472-8577-12c3e3b26100-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:15:19.537577 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.537553 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:15:19.545266 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:19.545240 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-159379b5-openshift-default-757bbb569b-lmdx8"] Jul 24 18:15:20.249156 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:15:20.249121 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" path="/var/lib/kubelet/pods/39a28f20-ed73-4472-8577-12c3e3b26100/volumes" Jul 24 18:16:04.440451 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:04.440350 2566 scope.go:117] "RemoveContainer" containerID="b2e3bd63461fb36c216b8fb5232bbfb2f28c27b6f529a72bd53434b9a259833f" Jul 24 18:16:11.497135 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497095 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:16:11.497694 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497678 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" containerName="payload-processing" Jul 24 18:16:11.497755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497697 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" containerName="payload-processing" Jul 24 18:16:11.497755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497713 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:11.497755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497722 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:11.497755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497731 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:11.497755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497740 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:11.497926 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497814 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:11.497926 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.497827 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="844cd73f-63a8-4f3b-a7a0-046a7fa1e141" containerName="payload-processing" Jul 24 18:16:11.500348 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.500331 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.503687 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.503661 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-04ecb2-openshift-default-dockercfg-z7vm9\"" Jul 24 18:16:11.503687 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.503662 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:16:11.516190 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.516159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:16:11.597523 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597523 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597527 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpx2m\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597923 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597923 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.597923 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.597861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698569 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698752 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698648 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698752 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698752 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpx2m\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698752 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698784 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.698973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.698874 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.699213 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.699107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.699429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.699399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.699429 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.699418 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.699666 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.699646 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.701078 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.701057 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.701557 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.701537 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.724762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.724733 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.725634 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.725612 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpx2m\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m\") pod \"e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.810784 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.810687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:11.960939 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:16:11.960910 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d6f983_7bbf_478c_b4f8_35472f888646.slice/crio-7906684c6b847cd06fb40755a060574f7a7ae99336acd41a779bed905909054b WatchSource:0}: Error finding container 7906684c6b847cd06fb40755a060574f7a7ae99336acd41a779bed905909054b: Status 404 returned error can't find the container with id 7906684c6b847cd06fb40755a060574f7a7ae99336acd41a779bed905909054b Jul 24 18:16:11.961654 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.961629 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:16:11.963168 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.963131 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:11.963226 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.963212 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:11.963262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:11.963252 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:12.794299 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:12.794262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerStarted","Data":"43c588e821fd38d83278a9cbb2671e9dcb3257f26bf069f56a23f4ff52b15ab0"} Jul 24 18:16:12.794729 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:12.794315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerStarted","Data":"7906684c6b847cd06fb40755a060574f7a7ae99336acd41a779bed905909054b"} Jul 24 18:16:12.811223 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:12.811184 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:12.819995 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:12.819941 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podStartSLOduration=1.819923967 podStartE2EDuration="1.819923967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:12.817450619 +0000 UTC m=+1989.018683008" watchObservedRunningTime="2026-07-24 18:16:12.819923967 +0000 UTC m=+1989.021156356" Jul 24 18:16:13.816817 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:13.816785 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:14.805334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:14.805304 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:14.806174 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:14.806157 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:15.680672 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.680631 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:16:15.681084 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.681071 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a28f20-ed73-4472-8577-12c3e3b26100" containerName="istio-proxy" Jul 24 18:16:15.683200 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.683168 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.686040 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.686020 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-04ecb2\"" Jul 24 18:16:15.686208 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.686191 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-04ecb2-dockercfg-nhv67\"" Jul 24 18:16:15.700192 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.700147 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:16:15.848741 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.848703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdw7d\" (UniqueName: \"kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.848903 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.848810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.950281 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.950171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdw7d\" (UniqueName: \"kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.950509 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.950313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.951058 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.951035 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.962283 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.962252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdw7d\" (UniqueName: \"kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d\") pod \"payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:15.994186 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:15.994146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:16.335639 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:16.335614 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:16:16.337571 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:16:16.337536 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cee7c32_0f07_4594_8eee_1d2f9c47eb4e.slice/crio-c677c5854b396d5bba0abf9b82d2a80e8ac64123c6a79c1afdebbadece193a87 WatchSource:0}: Error finding container c677c5854b396d5bba0abf9b82d2a80e8ac64123c6a79c1afdebbadece193a87: Status 404 returned error can't find the container with id c677c5854b396d5bba0abf9b82d2a80e8ac64123c6a79c1afdebbadece193a87 Jul 24 18:16:16.817544 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:16.817507 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" event={"ID":"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e","Type":"ContainerStarted","Data":"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c"} Jul 24 18:16:16.817955 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:16.817563 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" event={"ID":"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e","Type":"ContainerStarted","Data":"c677c5854b396d5bba0abf9b82d2a80e8ac64123c6a79c1afdebbadece193a87"} Jul 24 18:16:16.836409 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:16.836332 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" podStartSLOduration=1.836314837 podStartE2EDuration="1.836314837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:16.834564451 +0000 UTC m=+1993.035796841" watchObservedRunningTime="2026-07-24 18:16:16.836314837 +0000 UTC m=+1993.037547278" Jul 24 18:16:17.823239 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:17.823202 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:24.191890 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.191848 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:16:24.195287 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.195252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.199629 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.199597 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-dd7b67\"" Jul 24 18:16:24.219062 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.219029 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:16:24.230294 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.230251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.230448 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.230372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctw5d\" (UniqueName: \"kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.269720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.269687 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:16:24.272178 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.272163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.277266 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.277244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-dd7b67-dockercfg-zxmsq\"" Jul 24 18:16:24.310399 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.310346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:16:24.330913 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.330877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.331105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.330926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.331105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.330978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctw5d\" (UniqueName: \"kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.331105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.331015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spts2\" (UniqueName: \"kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.331919 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.331895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.343096 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.343064 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctw5d\" (UniqueName: \"kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d\") pod \"payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.432389 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.432324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.432588 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.432422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spts2\" (UniqueName: \"kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.433101 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.433076 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.442761 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.442704 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spts2\" (UniqueName: \"kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2\") pod \"payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.508570 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.508525 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.607441 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.607403 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.668775 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.668740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:16:24.672520 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:16:24.672475 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3941494_3a1e_49b9_ba23_1016fa0838a5.slice/crio-fc6affe75310ab77808ceb361bc5646bf9e300d09294b72e3aaaeeff2f5d51c4 WatchSource:0}: Error finding container fc6affe75310ab77808ceb361bc5646bf9e300d09294b72e3aaaeeff2f5d51c4: Status 404 returned error can't find the container with id fc6affe75310ab77808ceb361bc5646bf9e300d09294b72e3aaaeeff2f5d51c4 Jul 24 18:16:24.790092 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.790065 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:16:24.793105 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:16:24.793076 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f05c87_85e9_4149_859f_b9fa44bb2196.slice/crio-2df3bf20729880a5a6954c8bde0e63510be1e5c75074ff7f10dda42d6c0971c3 WatchSource:0}: Error finding container 2df3bf20729880a5a6954c8bde0e63510be1e5c75074ff7f10dda42d6c0971c3: Status 404 returned error can't find the container with id 2df3bf20729880a5a6954c8bde0e63510be1e5c75074ff7f10dda42d6c0971c3 Jul 24 18:16:24.870472 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.870434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" event={"ID":"d3941494-3a1e-49b9-ba23-1016fa0838a5","Type":"ContainerStarted","Data":"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984"} Jul 24 18:16:24.870644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.870496 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" event={"ID":"d3941494-3a1e-49b9-ba23-1016fa0838a5","Type":"ContainerStarted","Data":"fc6affe75310ab77808ceb361bc5646bf9e300d09294b72e3aaaeeff2f5d51c4"} Jul 24 18:16:24.870644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.870526 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:16:24.872599 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.872530 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" event={"ID":"e2f05c87-85e9-4149-859f-b9fa44bb2196","Type":"ContainerStarted","Data":"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2"} Jul 24 18:16:24.872599 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.872599 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:24.872803 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.872623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" event={"ID":"e2f05c87-85e9-4149-859f-b9fa44bb2196","Type":"ContainerStarted","Data":"2df3bf20729880a5a6954c8bde0e63510be1e5c75074ff7f10dda42d6c0971c3"} Jul 24 18:16:24.898299 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.898239 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" podStartSLOduration=0.898221954 podStartE2EDuration="898.221954ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:24.896225717 +0000 UTC m=+2001.097458107" watchObservedRunningTime="2026-07-24 18:16:24.898221954 +0000 UTC m=+2001.099454343" Jul 24 18:16:24.922264 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:24.922199 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" podStartSLOduration=0.922178236 podStartE2EDuration="922.178236ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:24.920566618 +0000 UTC m=+2001.121799029" watchObservedRunningTime="2026-07-24 18:16:24.922178236 +0000 UTC m=+2001.123410624" Jul 24 18:16:27.893731 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.893645 2566 generic.go:356] "Generic (PLEG): container finished" podID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerID="43c588e821fd38d83278a9cbb2671e9dcb3257f26bf069f56a23f4ff52b15ab0" exitCode=0 Jul 24 18:16:27.894132 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.893724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerDied","Data":"43c588e821fd38d83278a9cbb2671e9dcb3257f26bf069f56a23f4ff52b15ab0"} Jul 24 18:16:27.894177 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.894146 2566 scope.go:117] "RemoveContainer" containerID="43c588e821fd38d83278a9cbb2671e9dcb3257f26bf069f56a23f4ff52b15ab0" Jul 24 18:16:27.895209 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.895183 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:27.895271 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.895252 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:27.895310 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:27.895284 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:16:28.831886 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:28.831854 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:16:28.906750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:28.906718 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerStarted","Data":"a715fea0587ea60a49a2843201ab23336d5b896f178be07f6c4f18610e57d324"} Jul 24 18:16:29.806273 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:29.806229 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:29.811449 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:29.811419 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:29.812710 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:29.812676 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 24 18:16:29.812835 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:29.812731 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 24 18:16:30.812061 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:30.812025 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 24 18:16:30.812488 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:30.812095 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 24 18:16:31.811099 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:31.811061 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 24 18:16:31.811286 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:31.811122 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 24 18:16:32.811409 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:32.811374 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 24 18:16:32.811900 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:32.811475 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 24 18:16:33.811439 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:33.811340 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 24 18:16:33.811439 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:33.811435 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 24 18:16:34.815229 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:34.815203 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:34.816223 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:34.816202 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:16:35.881292 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:35.881246 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:16:35.881848 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:16:35.881445 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:17:06.851408 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.851351 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:17:06.855274 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.855251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.859431 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.859406 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dd7b67\"/\"openshift-service-ca.crt\"" Jul 24 18:17:06.859671 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.859496 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dd7b67\"/\"default-dockercfg-bp47c\"" Jul 24 18:17:06.859972 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.859552 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-dd7b67\"/\"test-model-dd7b67-kserve-self-signed-certs\"" Jul 24 18:17:06.860111 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.860024 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-dd7b67\"/\"kube-root-ca.crt\"" Jul 24 18:17:06.863062 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.863040 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:17:06.914079 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.914312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.914312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.914312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.914312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914170 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:06.914312 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:06.914258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk64b\" (UniqueName: \"kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.014854 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.014812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.014872 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk64b\" (UniqueName: \"kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.014943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.014978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.014995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015067 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.015017 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015347 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.015320 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015451 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.015405 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.015518 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.015490 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.018179 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.018149 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.018604 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.018582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.023769 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.023743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk64b\" (UniqueName: \"kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b\") pod \"test-model-dd7b67-kserve-96d785d9c-2qs8l\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.166899 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.166800 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:07.310576 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:07.310544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:17:07.312765 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:17:07.312734 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7160420_bb1d_4409_91e7_198a163df585.slice/crio-a3e1ed25cd1da2054c1dfdc684fd704913c5ccf87ad4a8f0cb44296cc268b71a WatchSource:0}: Error finding container a3e1ed25cd1da2054c1dfdc684fd704913c5ccf87ad4a8f0cb44296cc268b71a: Status 404 returned error can't find the container with id a3e1ed25cd1da2054c1dfdc684fd704913c5ccf87ad4a8f0cb44296cc268b71a Jul 24 18:17:08.129189 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:08.129142 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" event={"ID":"f7160420-bb1d-4409-91e7-198a163df585","Type":"ContainerStarted","Data":"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb"} Jul 24 18:17:08.129737 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:08.129203 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" event={"ID":"f7160420-bb1d-4409-91e7-198a163df585","Type":"ContainerStarted","Data":"a3e1ed25cd1da2054c1dfdc684fd704913c5ccf87ad4a8f0cb44296cc268b71a"} Jul 24 18:17:08.150919 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:08.150860 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" podStartSLOduration=2.150842124 podStartE2EDuration="2.150842124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:17:08.14842238 +0000 UTC m=+2044.349654777" watchObservedRunningTime="2026-07-24 18:17:08.150842124 +0000 UTC m=+2044.352074512" Jul 24 18:17:17.167452 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:17.167416 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:17.167934 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:17.167491 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:17.183300 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:17.182918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:17.197543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:17.197508 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:17:53.486057 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:17:53.486015 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:18:04.085011 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.084974 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:18:04.085517 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.085224 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" podUID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" containerName="payload-processing" containerID="cri-o://8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c" gracePeriod=30 Jul 24 18:18:04.335465 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.335391 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:18:04.435545 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.435506 2566 generic.go:356] "Generic (PLEG): container finished" podID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" containerID="8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c" exitCode=0 Jul 24 18:18:04.435725 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.435566 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" Jul 24 18:18:04.435725 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.435592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" event={"ID":"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e","Type":"ContainerDied","Data":"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c"} Jul 24 18:18:04.435725 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.435669 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd" event={"ID":"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e","Type":"ContainerDied","Data":"c677c5854b396d5bba0abf9b82d2a80e8ac64123c6a79c1afdebbadece193a87"} Jul 24 18:18:04.435725 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.435690 2566 scope.go:117] "RemoveContainer" containerID="8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c" Jul 24 18:18:04.447259 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.447230 2566 scope.go:117] "RemoveContainer" containerID="8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c" Jul 24 18:18:04.447483 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.447463 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume\") pod \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " Jul 24 18:18:04.447569 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.447551 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdw7d\" (UniqueName: \"kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d\") pod \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\" (UID: \"1cee7c32-0f07-4594-8eee-1d2f9c47eb4e\") " Jul 24 18:18:04.447741 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:18:04.447716 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c\": container with ID starting with 8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c not found: ID does not exist" containerID="8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c" Jul 24 18:18:04.447826 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.447750 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c"} err="failed to get container status \"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c\": rpc error: code = NotFound desc = could not find container \"8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c\": container with ID starting with 8a0e7a90a4ae0ae58799bb6089291397d96b82c5be2041c51e34636a0e9c043c not found: ID does not exist" Jul 24 18:18:04.447878 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.447848 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" (UID: "1cee7c32-0f07-4594-8eee-1d2f9c47eb4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:18:04.449553 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.449531 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d" (OuterVolumeSpecName: "kube-api-access-kdw7d") pod "1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" (UID: "1cee7c32-0f07-4594-8eee-1d2f9c47eb4e"). InnerVolumeSpecName "kube-api-access-kdw7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:04.548991 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.548955 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:04.548991 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.548983 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdw7d\" (UniqueName: \"kubernetes.io/projected/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e-kube-api-access-kdw7d\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:04.757517 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.757482 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:18:04.760837 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:04.760809 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-04ecb2-79b5bc8558-w56sd"] Jul 24 18:18:06.249735 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:06.249704 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" path="/var/lib/kubelet/pods/1cee7c32-0f07-4594-8eee-1d2f9c47eb4e/volumes" Jul 24 18:18:08.506873 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.506835 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" podUID="f7160420-bb1d-4409-91e7-198a163df585" containerName="main" containerID="cri-o://fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb" gracePeriod=45 Jul 24 18:18:08.765048 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.764985 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:18:08.784207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784175 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.784422 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784226 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.784422 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784257 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.784422 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784331 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk64b\" (UniqueName: \"kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.784422 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784380 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.784422 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784411 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm\") pod \"f7160420-bb1d-4409-91e7-198a163df585\" (UID: \"f7160420-bb1d-4409-91e7-198a163df585\") " Jul 24 18:18:08.785038 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.784996 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:08.785297 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.785255 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home" (OuterVolumeSpecName: "home") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:08.785297 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.785248 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:08.787101 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.787077 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm" (OuterVolumeSpecName: "dshm") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:08.787235 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.787217 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b" (OuterVolumeSpecName: "kube-api-access-tk64b") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "kube-api-access-tk64b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:08.787297 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.787272 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7160420-bb1d-4409-91e7-198a163df585" (UID: "f7160420-bb1d-4409-91e7-198a163df585"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 18:18:08.885813 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885773 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk64b\" (UniqueName: \"kubernetes.io/projected/f7160420-bb1d-4409-91e7-198a163df585-kube-api-access-tk64b\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:08.885813 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885806 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-home\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:08.885813 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885815 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-dshm\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:08.886052 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885823 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-model-cache\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:08.886052 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885836 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7160420-bb1d-4409-91e7-198a163df585-tmp-dir\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:08.886052 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:08.885843 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7160420-bb1d-4409-91e7-198a163df585-tls-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:09.464859 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.464822 2566 generic.go:356] "Generic (PLEG): container finished" podID="f7160420-bb1d-4409-91e7-198a163df585" containerID="fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb" exitCode=0 Jul 24 18:18:09.465083 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.464866 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" Jul 24 18:18:09.465083 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.464883 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" event={"ID":"f7160420-bb1d-4409-91e7-198a163df585","Type":"ContainerDied","Data":"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb"} Jul 24 18:18:09.465083 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.464920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l" event={"ID":"f7160420-bb1d-4409-91e7-198a163df585","Type":"ContainerDied","Data":"a3e1ed25cd1da2054c1dfdc684fd704913c5ccf87ad4a8f0cb44296cc268b71a"} Jul 24 18:18:09.465083 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.464941 2566 scope.go:117] "RemoveContainer" containerID="fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb" Jul 24 18:18:09.474291 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.474273 2566 scope.go:117] "RemoveContainer" containerID="fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb" Jul 24 18:18:09.474578 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:18:09.474557 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb\": container with ID starting with fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb not found: ID does not exist" containerID="fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb" Jul 24 18:18:09.474665 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.474587 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb"} err="failed to get container status \"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb\": rpc error: code = NotFound desc = could not find container \"fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb\": container with ID starting with fcfb9c4ae587d5cc0f56cc0d66a0c34432b254daa166944d9a88b09a91e8d0cb not found: ID does not exist" Jul 24 18:18:09.492545 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.492507 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:18:09.497894 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:09.497861 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-dd7b67/test-model-dd7b67-kserve-96d785d9c-2qs8l"] Jul 24 18:18:10.249659 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:10.249625 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7160420-bb1d-4409-91e7-198a163df585" path="/var/lib/kubelet/pods/f7160420-bb1d-4409-91e7-198a163df585/volumes" Jul 24 18:18:15.287744 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:15.287711 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:18:15.288133 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:15.287936 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" containerID="cri-o://a715fea0587ea60a49a2843201ab23336d5b896f178be07f6c4f18610e57d324" gracePeriod=30 Jul 24 18:18:19.817801 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:19.817755 2566 patch_prober.go:28] interesting pod/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:18:19.818248 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:19.817829 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:18:20.526615 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.526584 2566 generic.go:356] "Generic (PLEG): container finished" podID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerID="a715fea0587ea60a49a2843201ab23336d5b896f178be07f6c4f18610e57d324" exitCode=0 Jul 24 18:18:20.526732 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.526657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerDied","Data":"a715fea0587ea60a49a2843201ab23336d5b896f178be07f6c4f18610e57d324"} Jul 24 18:18:20.526732 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.526729 2566 scope.go:117] "RemoveContainer" containerID="43c588e821fd38d83278a9cbb2671e9dcb3257f26bf069f56a23f4ff52b15ab0" Jul 24 18:18:20.543334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.543312 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:18:20.599720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599618 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.599720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599661 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.599720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599682 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.599720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599704 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599735 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpx2m\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599754 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599795 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599969 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:20.600071 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599981 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data" (OuterVolumeSpecName: "istio-data") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:20.600344 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600114 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:20.600344 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600127 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:20.600344 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.599902 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600344 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600194 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert\") pod \"a8d6f983-7bbf-478c-b4f8-35472f888646\" (UID: \"a8d6f983-7bbf-478c-b4f8-35472f888646\") " Jul 24 18:18:20.600565 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600549 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.600607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600573 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.600607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600586 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.600607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600550 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:18:20.600607 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.600598 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.602601 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.602579 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:18:20.602601 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.602586 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m" (OuterVolumeSpecName: "kube-api-access-lpx2m") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "kube-api-access-lpx2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:20.602719 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.602585 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:18:20.602719 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.602605 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token" (OuterVolumeSpecName: "istio-token") pod "a8d6f983-7bbf-478c-b4f8-35472f888646" (UID: "a8d6f983-7bbf-478c-b4f8-35472f888646"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:20.702085 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.702035 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.702085 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.702073 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8d6f983-7bbf-478c-b4f8-35472f888646-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.702085 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.702084 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.702085 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.702092 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpx2m\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-kube-api-access-lpx2m\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:20.702399 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:20.702103 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8d6f983-7bbf-478c-b4f8-35472f888646-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:21.533541 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:21.533508 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" Jul 24 18:18:21.534022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:21.533550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr" event={"ID":"a8d6f983-7bbf-478c-b4f8-35472f888646","Type":"ContainerDied","Data":"7906684c6b847cd06fb40755a060574f7a7ae99336acd41a779bed905909054b"} Jul 24 18:18:21.534022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:21.533600 2566 scope.go:117] "RemoveContainer" containerID="a715fea0587ea60a49a2843201ab23336d5b896f178be07f6c4f18610e57d324" Jul 24 18:18:21.568689 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:21.568652 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:18:21.572106 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:21.572076 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-04ecb2-openshift-default-75d7d64466-m99zr"] Jul 24 18:18:22.249973 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:22.249941 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" path="/var/lib/kubelet/pods/a8d6f983-7bbf-478c-b4f8-35472f888646/volumes" Jul 24 18:18:26.135786 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.135743 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:18:26.136255 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.135999 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" podUID="e2f05c87-85e9-4149-859f-b9fa44bb2196" containerName="payload-processing" containerID="cri-o://4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2" gracePeriod=30 Jul 24 18:18:26.141577 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.141550 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:18:26.141798 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.141770 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" podUID="d3941494-3a1e-49b9-ba23-1016fa0838a5" containerName="payload-pre-processing" containerID="cri-o://724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984" gracePeriod=30 Jul 24 18:18:26.403874 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.403848 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:18:26.407080 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.407059 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:18:26.551700 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.551653 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume\") pod \"e2f05c87-85e9-4149-859f-b9fa44bb2196\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " Jul 24 18:18:26.551700 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.551715 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spts2\" (UniqueName: \"kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2\") pod \"e2f05c87-85e9-4149-859f-b9fa44bb2196\" (UID: \"e2f05c87-85e9-4149-859f-b9fa44bb2196\") " Jul 24 18:18:26.551952 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.551795 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctw5d\" (UniqueName: \"kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d\") pod \"d3941494-3a1e-49b9-ba23-1016fa0838a5\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " Jul 24 18:18:26.551952 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.551828 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume\") pod \"d3941494-3a1e-49b9-ba23-1016fa0838a5\" (UID: \"d3941494-3a1e-49b9-ba23-1016fa0838a5\") " Jul 24 18:18:26.552233 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.552196 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2f05c87-85e9-4149-859f-b9fa44bb2196" (UID: "e2f05c87-85e9-4149-859f-b9fa44bb2196"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:18:26.552305 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.552232 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3941494-3a1e-49b9-ba23-1016fa0838a5" (UID: "d3941494-3a1e-49b9-ba23-1016fa0838a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:18:26.553915 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.553893 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2" (OuterVolumeSpecName: "kube-api-access-spts2") pod "e2f05c87-85e9-4149-859f-b9fa44bb2196" (UID: "e2f05c87-85e9-4149-859f-b9fa44bb2196"). InnerVolumeSpecName "kube-api-access-spts2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:26.553978 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.553927 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d" (OuterVolumeSpecName: "kube-api-access-ctw5d") pod "d3941494-3a1e-49b9-ba23-1016fa0838a5" (UID: "d3941494-3a1e-49b9-ba23-1016fa0838a5"). InnerVolumeSpecName "kube-api-access-ctw5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:18:26.562029 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.562003 2566 generic.go:356] "Generic (PLEG): container finished" podID="d3941494-3a1e-49b9-ba23-1016fa0838a5" containerID="724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984" exitCode=0 Jul 24 18:18:26.562149 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.562044 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" Jul 24 18:18:26.562149 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.562045 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" event={"ID":"d3941494-3a1e-49b9-ba23-1016fa0838a5","Type":"ContainerDied","Data":"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984"} Jul 24 18:18:26.562149 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.562101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm" event={"ID":"d3941494-3a1e-49b9-ba23-1016fa0838a5","Type":"ContainerDied","Data":"fc6affe75310ab77808ceb361bc5646bf9e300d09294b72e3aaaeeff2f5d51c4"} Jul 24 18:18:26.562149 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.562125 2566 scope.go:117] "RemoveContainer" containerID="724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984" Jul 24 18:18:26.563506 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.563482 2566 generic.go:356] "Generic (PLEG): container finished" podID="e2f05c87-85e9-4149-859f-b9fa44bb2196" containerID="4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2" exitCode=0 Jul 24 18:18:26.563615 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.563521 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" Jul 24 18:18:26.563615 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.563538 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" event={"ID":"e2f05c87-85e9-4149-859f-b9fa44bb2196","Type":"ContainerDied","Data":"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2"} Jul 24 18:18:26.563615 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.563582 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6" event={"ID":"e2f05c87-85e9-4149-859f-b9fa44bb2196","Type":"ContainerDied","Data":"2df3bf20729880a5a6954c8bde0e63510be1e5c75074ff7f10dda42d6c0971c3"} Jul 24 18:18:26.572556 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.572374 2566 scope.go:117] "RemoveContainer" containerID="724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984" Jul 24 18:18:26.572727 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:18:26.572705 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984\": container with ID starting with 724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984 not found: ID does not exist" containerID="724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984" Jul 24 18:18:26.572812 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.572739 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984"} err="failed to get container status \"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984\": rpc error: code = NotFound desc = could not find container \"724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984\": container with ID starting with 724dac6c236ae26fe87e9a5faef8713d2920aac20e35703bb376d2c0040ed984 not found: ID does not exist" Jul 24 18:18:26.572812 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.572762 2566 scope.go:117] "RemoveContainer" containerID="4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2" Jul 24 18:18:26.580999 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.580980 2566 scope.go:117] "RemoveContainer" containerID="4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2" Jul 24 18:18:26.581239 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:18:26.581217 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2\": container with ID starting with 4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2 not found: ID does not exist" containerID="4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2" Jul 24 18:18:26.581307 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.581245 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2"} err="failed to get container status \"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2\": rpc error: code = NotFound desc = could not find container \"4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2\": container with ID starting with 4d53899814d682fc09332557c9c6b337678c26f64c1a4e666c527f04a70c9eb2 not found: ID does not exist" Jul 24 18:18:26.589555 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.589533 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:18:26.593154 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.593134 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dd7b67-687b74888-w8ncm"] Jul 24 18:18:26.604986 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.604958 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:18:26.607022 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.607001 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dd7b67-d498f4ffb-m9ls6"] Jul 24 18:18:26.652818 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.652727 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctw5d\" (UniqueName: \"kubernetes.io/projected/d3941494-3a1e-49b9-ba23-1016fa0838a5-kube-api-access-ctw5d\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:26.652818 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.652770 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3941494-3a1e-49b9-ba23-1016fa0838a5-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:26.652818 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.652781 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f05c87-85e9-4149-859f-b9fa44bb2196-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:26.652818 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:26.652791 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spts2\" (UniqueName: \"kubernetes.io/projected/e2f05c87-85e9-4149-859f-b9fa44bb2196-kube-api-access-spts2\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:18:28.249189 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:28.249150 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3941494-3a1e-49b9-ba23-1016fa0838a5" path="/var/lib/kubelet/pods/d3941494-3a1e-49b9-ba23-1016fa0838a5/volumes" Jul 24 18:18:28.249568 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:28.249522 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f05c87-85e9-4149-859f-b9fa44bb2196" path="/var/lib/kubelet/pods/e2f05c87-85e9-4149-859f-b9fa44bb2196/volumes" Jul 24 18:18:38.220181 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220149 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220508 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" containerName="payload-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220520 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" containerName="payload-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220531 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220537 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220547 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220552 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220562 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f05c87-85e9-4149-859f-b9fa44bb2196" containerName="payload-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220567 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f05c87-85e9-4149-859f-b9fa44bb2196" containerName="payload-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220574 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3941494-3a1e-49b9-ba23-1016fa0838a5" containerName="payload-pre-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220580 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3941494-3a1e-49b9-ba23-1016fa0838a5" containerName="payload-pre-processing" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220591 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7160420-bb1d-4409-91e7-198a163df585" containerName="main" Jul 24 18:18:38.220628 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220598 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7160420-bb1d-4409-91e7-198a163df585" containerName="main" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220671 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220682 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3941494-3a1e-49b9-ba23-1016fa0838a5" containerName="payload-pre-processing" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220691 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f05c87-85e9-4149-859f-b9fa44bb2196" containerName="payload-processing" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220711 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7160420-bb1d-4409-91e7-198a163df585" containerName="main" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220722 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cee7c32-0f07-4594-8eee-1d2f9c47eb4e" containerName="payload-processing" Jul 24 18:18:38.221103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.220732 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d6f983-7bbf-478c-b4f8-35472f888646" containerName="istio-proxy" Jul 24 18:18:38.224199 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.224179 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.226896 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.226876 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ab4b1a-openshift-default-dockercfg-c62qr\"" Jul 24 18:18:38.227006 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.226985 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:18:38.236624 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.236601 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:18:38.356093 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356298 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356123 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356298 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356463 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356463 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356423 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356584 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnklh\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356643 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356745 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.356828 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.356809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457736 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457749 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457784 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457816 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.457924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.457970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnklh\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458108 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458174 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458276 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458272 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458555 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458426 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.458744 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.458725 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.460805 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.460780 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.460999 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.460980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.466162 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.466138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.466274 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.466259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnklh\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh\") pod \"e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.536338 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.536237 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:38.677918 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.677889 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:18:38.679282 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:18:38.679254 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae5497f_5838_46dc_9678_20b9b03cffea.slice/crio-67ef06005320ba92279eda8cb6877ee6e3f297cfd0fa0f938948377db15ff773 WatchSource:0}: Error finding container 67ef06005320ba92279eda8cb6877ee6e3f297cfd0fa0f938948377db15ff773: Status 404 returned error can't find the container with id 67ef06005320ba92279eda8cb6877ee6e3f297cfd0fa0f938948377db15ff773 Jul 24 18:18:38.681632 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.681588 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:38.681717 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.681683 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:38.681759 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:38.681723 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:39.645197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:39.645156 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerStarted","Data":"3ab908322fcd5073e0734fd4f627334a616d6e28b2f5d005eaf0de9af602dc5b"} Jul 24 18:18:39.645591 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:39.645223 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerStarted","Data":"67ef06005320ba92279eda8cb6877ee6e3f297cfd0fa0f938948377db15ff773"} Jul 24 18:18:39.665513 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:39.665457 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podStartSLOduration=1.6654407230000001 podStartE2EDuration="1.665440723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:18:39.664707357 +0000 UTC m=+2135.865939729" watchObservedRunningTime="2026-07-24 18:18:39.665440723 +0000 UTC m=+2135.866673112" Jul 24 18:18:40.536690 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:40.536653 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:40.541304 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:40.541279 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:40.650411 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:40.650382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:40.651405 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:40.651385 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:42.842070 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.842035 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:18:42.844559 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.844540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:42.849343 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.849317 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ab4b1a\"" Jul 24 18:18:42.849456 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.849325 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ab4b1a-dockercfg-xwz5n\"" Jul 24 18:18:42.855932 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.855906 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:18:43.000003 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:42.999961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.000191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.000027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjcxg\" (UniqueName: \"kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.101270 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.101239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjcxg\" (UniqueName: \"kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.101501 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.101353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.102217 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.102193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.117021 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.116996 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjcxg\" (UniqueName: \"kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg\") pod \"payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.156931 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.156894 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.298109 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.298076 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:18:43.299436 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:18:43.299352 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83deb945_ca4c_45c8_b7fe_9485a8b3f23b.slice/crio-ab40e9a73e3cafa2df1a426510f77b0a9862cb1c7e0f9b40f8eaec83abab5860 WatchSource:0}: Error finding container ab40e9a73e3cafa2df1a426510f77b0a9862cb1c7e0f9b40f8eaec83abab5860: Status 404 returned error can't find the container with id ab40e9a73e3cafa2df1a426510f77b0a9862cb1c7e0f9b40f8eaec83abab5860 Jul 24 18:18:43.669247 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.669147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" event={"ID":"83deb945-ca4c-45c8-b7fe-9485a8b3f23b","Type":"ContainerStarted","Data":"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670"} Jul 24 18:18:43.669247 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.669206 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" event={"ID":"83deb945-ca4c-45c8-b7fe-9485a8b3f23b","Type":"ContainerStarted","Data":"ab40e9a73e3cafa2df1a426510f77b0a9862cb1c7e0f9b40f8eaec83abab5860"} Jul 24 18:18:43.669516 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.669261 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:43.687968 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:43.687911 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" podStartSLOduration=1.687896189 podStartE2EDuration="1.687896189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:18:43.686016171 +0000 UTC m=+2139.887248562" watchObservedRunningTime="2026-07-24 18:18:43.687896189 +0000 UTC m=+2139.889128578" Jul 24 18:18:50.451097 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.451060 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:18:50.453943 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.453923 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.456604 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.456581 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ac2de9\"" Jul 24 18:18:50.456715 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.456617 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-ac2de9-dockercfg-26dpc\"" Jul 24 18:18:50.465970 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.465945 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:18:50.469116 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.469094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcms\" (UniqueName: \"kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.469219 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.469139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.569608 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.569571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcms\" (UniqueName: \"kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.569797 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.569642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.570435 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.570403 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.593774 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.593738 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcms\" (UniqueName: \"kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms\") pod \"payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.765166 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.765071 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:50.972269 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:50.972242 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:18:50.973912 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:18:50.973886 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1e1e00_909e_4ce3_95a8_9273f64bfd16.slice/crio-811065a53b7ac1bc2953d872ee4a42f5b8cfabfb2a82d42400b1d700948b5a26 WatchSource:0}: Error finding container 811065a53b7ac1bc2953d872ee4a42f5b8cfabfb2a82d42400b1d700948b5a26: Status 404 returned error can't find the container with id 811065a53b7ac1bc2953d872ee4a42f5b8cfabfb2a82d42400b1d700948b5a26 Jul 24 18:18:51.722297 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:51.722253 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" event={"ID":"eb1e1e00-909e-4ce3-95a8-9273f64bfd16","Type":"ContainerStarted","Data":"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c"} Jul 24 18:18:51.722836 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:51.722311 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" event={"ID":"eb1e1e00-909e-4ce3-95a8-9273f64bfd16","Type":"ContainerStarted","Data":"811065a53b7ac1bc2953d872ee4a42f5b8cfabfb2a82d42400b1d700948b5a26"} Jul 24 18:18:51.722836 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:51.722410 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:18:51.743842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:51.743790 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" podStartSLOduration=1.743772618 podStartE2EDuration="1.743772618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:18:51.742101766 +0000 UTC m=+2147.943334155" watchObservedRunningTime="2026-07-24 18:18:51.743772618 +0000 UTC m=+2147.945005007" Jul 24 18:18:54.677177 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:54.677142 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:18:55.652236 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.652149 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:18:55.652236 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.652207 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:18:55.746853 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.746774 2566 generic.go:356] "Generic (PLEG): container finished" podID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerID="3ab908322fcd5073e0734fd4f627334a616d6e28b2f5d005eaf0de9af602dc5b" exitCode=0 Jul 24 18:18:55.747210 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.746845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerDied","Data":"3ab908322fcd5073e0734fd4f627334a616d6e28b2f5d005eaf0de9af602dc5b"} Jul 24 18:18:55.747254 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.747221 2566 scope.go:117] "RemoveContainer" containerID="3ab908322fcd5073e0734fd4f627334a616d6e28b2f5d005eaf0de9af602dc5b" Jul 24 18:18:55.748177 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.748149 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:55.748262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.748214 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:55.748262 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:55.748245 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:18:56.754351 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:56.754278 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerStarted","Data":"fa4ffb182a8eea9c07b2d1949828cd3de6b3da870ca97df753d1c400eaa9f4f2"} Jul 24 18:18:57.536426 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:57.536385 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:18:57.536757 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:57.536731 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:18:57.536870 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:57.536792 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:18:58.537205 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:58.537171 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:18:58.537610 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:58.537228 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:18:59.536687 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:59.536647 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:18:59.536874 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:18:59.536713 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:19:00.536974 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:00.536938 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:19:00.537400 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:00.536992 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:19:01.537507 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:01.537469 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 24 18:19:01.537982 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:01.537567 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 24 18:19:02.541159 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:02.541125 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:19:02.541547 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:02.541381 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:19:02.542230 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:02.542214 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:19:02.729951 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:02.729915 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:19:18.070348 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.070310 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:19:18.071242 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.071195 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" podUID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" containerName="payload-processing" containerID="cri-o://326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670" gracePeriod=30 Jul 24 18:19:18.329775 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.329714 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:19:18.440370 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.440321 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjcxg\" (UniqueName: \"kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg\") pod \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " Jul 24 18:19:18.440620 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.440397 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume\") pod \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\" (UID: \"83deb945-ca4c-45c8-b7fe-9485a8b3f23b\") " Jul 24 18:19:18.440825 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.440796 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83deb945-ca4c-45c8-b7fe-9485a8b3f23b" (UID: "83deb945-ca4c-45c8-b7fe-9485a8b3f23b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:19:18.442477 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.442453 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg" (OuterVolumeSpecName: "kube-api-access-zjcxg") pod "83deb945-ca4c-45c8-b7fe-9485a8b3f23b" (UID: "83deb945-ca4c-45c8-b7fe-9485a8b3f23b"). InnerVolumeSpecName "kube-api-access-zjcxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:18.542039 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.542000 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:18.542039 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.542029 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjcxg\" (UniqueName: \"kubernetes.io/projected/83deb945-ca4c-45c8-b7fe-9485a8b3f23b-kube-api-access-zjcxg\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:18.874898 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.874860 2566 generic.go:356] "Generic (PLEG): container finished" podID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" containerID="326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670" exitCode=0 Jul 24 18:19:18.875077 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.874917 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" Jul 24 18:19:18.875077 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.874948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" event={"ID":"83deb945-ca4c-45c8-b7fe-9485a8b3f23b","Type":"ContainerDied","Data":"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670"} Jul 24 18:19:18.875077 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.875006 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr" event={"ID":"83deb945-ca4c-45c8-b7fe-9485a8b3f23b","Type":"ContainerDied","Data":"ab40e9a73e3cafa2df1a426510f77b0a9862cb1c7e0f9b40f8eaec83abab5860"} Jul 24 18:19:18.875077 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.875044 2566 scope.go:117] "RemoveContainer" containerID="326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670" Jul 24 18:19:18.884063 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.884044 2566 scope.go:117] "RemoveContainer" containerID="326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670" Jul 24 18:19:18.884322 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:19:18.884300 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670\": container with ID starting with 326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670 not found: ID does not exist" containerID="326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670" Jul 24 18:19:18.884404 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.884330 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670"} err="failed to get container status \"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670\": rpc error: code = NotFound desc = could not find container \"326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670\": container with ID starting with 326f615604b162ab7b50544b749c37e9d8a34fc776cace8f5717d68b034c9670 not found: ID does not exist" Jul 24 18:19:18.898213 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.898190 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:19:18.902152 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:18.902128 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ab4b1a-976bbb586-g72gr"] Jul 24 18:19:20.249408 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:20.249377 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" path="/var/lib/kubelet/pods/83deb945-ca4c-45c8-b7fe-9485a8b3f23b/volumes" Jul 24 18:19:28.749194 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:28.749150 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:19:28.749660 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:28.749372 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" containerID="cri-o://fa4ffb182a8eea9c07b2d1949828cd3de6b3da870ca97df753d1c400eaa9f4f2" gracePeriod=30 Jul 24 18:19:32.543947 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:32.543904 2566 patch_prober.go:28] interesting pod/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:19:32.544383 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:32.543982 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:19:33.960668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:33.960634 2566 generic.go:356] "Generic (PLEG): container finished" podID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerID="fa4ffb182a8eea9c07b2d1949828cd3de6b3da870ca97df753d1c400eaa9f4f2" exitCode=0 Jul 24 18:19:33.961143 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:33.960714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerDied","Data":"fa4ffb182a8eea9c07b2d1949828cd3de6b3da870ca97df753d1c400eaa9f4f2"} Jul 24 18:19:33.961143 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:33.960781 2566 scope.go:117] "RemoveContainer" containerID="3ab908322fcd5073e0734fd4f627334a616d6e28b2f5d005eaf0de9af602dc5b" Jul 24 18:19:34.006889 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.006855 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:19:34.081926 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081827 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.081926 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081878 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.081926 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081897 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081940 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081975 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.081990 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnklh\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082012 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082037 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082197 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082070 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert\") pod \"8ae5497f-5838-46dc-9678-20b9b03cffea\" (UID: \"8ae5497f-5838-46dc-9678-20b9b03cffea\") " Jul 24 18:19:34.082529 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082204 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:34.082529 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082316 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:34.082529 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082491 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.082529 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082513 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.082745 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082529 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:34.082745 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082579 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:19:34.082745 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.082709 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:34.084289 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.084259 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:19:34.084481 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.084334 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:19:34.084564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.084532 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh" (OuterVolumeSpecName: "kube-api-access-jnklh") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "kube-api-access-jnklh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:34.084612 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.084555 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ae5497f-5838-46dc-9678-20b9b03cffea" (UID: "8ae5497f-5838-46dc-9678-20b9b03cffea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:34.183176 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183128 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183176 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183166 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183176 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183179 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183483 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183190 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnklh\" (UniqueName: \"kubernetes.io/projected/8ae5497f-5838-46dc-9678-20b9b03cffea-kube-api-access-jnklh\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183483 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183205 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183483 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183217 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ae5497f-5838-46dc-9678-20b9b03cffea-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.183483 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.183227 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ae5497f-5838-46dc-9678-20b9b03cffea-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:34.967630 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.967595 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" Jul 24 18:19:34.968138 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.967629 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k" event={"ID":"8ae5497f-5838-46dc-9678-20b9b03cffea","Type":"ContainerDied","Data":"67ef06005320ba92279eda8cb6877ee6e3f297cfd0fa0f938948377db15ff773"} Jul 24 18:19:34.968138 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.967694 2566 scope.go:117] "RemoveContainer" containerID="fa4ffb182a8eea9c07b2d1949828cd3de6b3da870ca97df753d1c400eaa9f4f2" Jul 24 18:19:34.988571 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.988539 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:19:34.991842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:34.991817 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ab4b1a-openshift-default-575994464b-zdq4k"] Jul 24 18:19:36.249268 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:36.249235 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" path="/var/lib/kubelet/pods/8ae5497f-5838-46dc-9678-20b9b03cffea/volumes" Jul 24 18:19:44.669527 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.669482 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:19:44.669997 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.669750 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" podUID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" containerName="payload-processing" containerID="cri-o://6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c" gracePeriod=30 Jul 24 18:19:44.925575 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.925507 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:19:44.985820 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.985785 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume\") pod \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " Jul 24 18:19:44.986012 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.985859 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgcms\" (UniqueName: \"kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms\") pod \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\" (UID: \"eb1e1e00-909e-4ce3-95a8-9273f64bfd16\") " Jul 24 18:19:44.986150 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.986128 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb1e1e00-909e-4ce3-95a8-9273f64bfd16" (UID: "eb1e1e00-909e-4ce3-95a8-9273f64bfd16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:19:44.986221 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.986214 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:44.987920 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:44.987888 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms" (OuterVolumeSpecName: "kube-api-access-dgcms") pod "eb1e1e00-909e-4ce3-95a8-9273f64bfd16" (UID: "eb1e1e00-909e-4ce3-95a8-9273f64bfd16"). InnerVolumeSpecName "kube-api-access-dgcms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:19:45.021213 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.021178 2566 generic.go:356] "Generic (PLEG): container finished" podID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" containerID="6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c" exitCode=0 Jul 24 18:19:45.021403 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.021225 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" Jul 24 18:19:45.021403 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.021264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" event={"ID":"eb1e1e00-909e-4ce3-95a8-9273f64bfd16","Type":"ContainerDied","Data":"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c"} Jul 24 18:19:45.021403 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.021320 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw" event={"ID":"eb1e1e00-909e-4ce3-95a8-9273f64bfd16","Type":"ContainerDied","Data":"811065a53b7ac1bc2953d872ee4a42f5b8cfabfb2a82d42400b1d700948b5a26"} Jul 24 18:19:45.021403 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.021348 2566 scope.go:117] "RemoveContainer" containerID="6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c" Jul 24 18:19:45.030468 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.030450 2566 scope.go:117] "RemoveContainer" containerID="6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c" Jul 24 18:19:45.030752 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:19:45.030734 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c\": container with ID starting with 6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c not found: ID does not exist" containerID="6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c" Jul 24 18:19:45.030795 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.030764 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c"} err="failed to get container status \"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c\": rpc error: code = NotFound desc = could not find container \"6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c\": container with ID starting with 6a2c2997c11ff275dee8b33ab7dddf048c6051c4fd938dabb91e020de57faf1c not found: ID does not exist" Jul 24 18:19:45.045351 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.045319 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:19:45.047489 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.047465 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ac2de9-7899b74854-jjcdw"] Jul 24 18:19:45.087241 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:45.087201 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgcms\" (UniqueName: \"kubernetes.io/projected/eb1e1e00-909e-4ce3-95a8-9273f64bfd16-kube-api-access-dgcms\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:19:46.249378 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:46.249328 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" path="/var/lib/kubelet/pods/eb1e1e00-909e-4ce3-95a8-9273f64bfd16/volumes" Jul 24 18:19:56.764139 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764103 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764486 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" containerName="payload-processing" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764498 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" containerName="payload-processing" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764509 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" containerName="payload-processing" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764514 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" containerName="payload-processing" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764535 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764541 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764604 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="83deb945-ca4c-45c8-b7fe-9485a8b3f23b" containerName="payload-processing" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764612 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764620 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:19:56.764626 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.764628 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1e1e00-909e-4ce3-95a8-9273f64bfd16" containerName="payload-processing" Jul 24 18:19:56.767509 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.767490 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.770669 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.770649 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2542df-openshift-default-dockercfg-d4zhn\"" Jul 24 18:19:56.771284 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.771265 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 18:19:56.787087 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.787047 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:19:56.900448 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900448 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59plw\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900641 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:56.900908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:56.900784 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002139 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002139 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002457 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002457 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002194 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59plw\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002457 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002457 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002670 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002670 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002670 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002670 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002602 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002886 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002940 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002892 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002999 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002933 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.002999 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.002938 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.004738 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.004696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.004960 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.004937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.010953 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.010930 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.011332 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.011314 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59plw\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw\") pod \"e2e-shared-a-2542df-openshift-default-6b967b6764-f559r\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.077521 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.077423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:57.214988 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.214959 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:19:57.216766 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:19:57.216719 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6487aa76_6978_46bc_bcb9_0c6268377206.slice/crio-97e5d4095b22e83f0160fca906f1b2d2fa6f896c7f750346f789f1d15309d692 WatchSource:0}: Error finding container 97e5d4095b22e83f0160fca906f1b2d2fa6f896c7f750346f789f1d15309d692: Status 404 returned error can't find the container with id 97e5d4095b22e83f0160fca906f1b2d2fa6f896c7f750346f789f1d15309d692 Jul 24 18:19:57.218910 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.218877 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:57.218995 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.218944 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:57.218995 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:57.218979 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:19:58.089334 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:58.089298 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerStarted","Data":"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591"} Jul 24 18:19:58.089751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:58.089351 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerStarted","Data":"97e5d4095b22e83f0160fca906f1b2d2fa6f896c7f750346f789f1d15309d692"} Jul 24 18:19:58.112143 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:58.112090 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podStartSLOduration=2.112074399 podStartE2EDuration="2.112074399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:19:58.110614577 +0000 UTC m=+2214.311846957" watchObservedRunningTime="2026-07-24 18:19:58.112074399 +0000 UTC m=+2214.313306786" Jul 24 18:19:59.078533 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:59.078488 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:59.083511 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:59.083482 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:59.095105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:59.095078 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:19:59.095968 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:19:59.095950 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:05.364507 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.364468 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:20:05.364927 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.364855 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:20:05.364927 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.364868 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ae5497f-5838-46dc-9678-20b9b03cffea" containerName="istio-proxy" Jul 24 18:20:05.367716 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.367693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.371855 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.371830 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f556c1-openshift-default-dockercfg-x6v4q\"" Jul 24 18:20:05.405390 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.404744 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:20:05.476017 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.475978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-825t7\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476070 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476096 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476434 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476434 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476434 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.476560 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.476438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.577866 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.577815 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.578668 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.578645 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.579267 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.579249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825t7\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.579847 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.579795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.580033 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.580017 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.580161 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.580147 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.580306 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.580291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581055 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.581031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581178 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.581164 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581329 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.581230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581448 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.579136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581555 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.581530 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.581977 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.581943 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.584610 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.582171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.585106 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.585078 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.588227 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.587082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.589033 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.588984 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-825t7\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.590491 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.590468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token\") pod \"e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.678799 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.678677 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:05.861869 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.861830 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:20:05.863281 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:20:05.863241 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf385ed_801e_4124_9343_1496a6b0bf24.slice/crio-21563f195703ac7a1e78bd816b5c51da7f339a655e1d0c1999817a0bc4bf7520 WatchSource:0}: Error finding container 21563f195703ac7a1e78bd816b5c51da7f339a655e1d0c1999817a0bc4bf7520: Status 404 returned error can't find the container with id 21563f195703ac7a1e78bd816b5c51da7f339a655e1d0c1999817a0bc4bf7520 Jul 24 18:20:05.865586 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.865547 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:05.865665 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.865619 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:05.865665 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:05.865651 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:06.141838 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:06.141803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerStarted","Data":"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee"} Jul 24 18:20:06.142156 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:06.141854 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerStarted","Data":"21563f195703ac7a1e78bd816b5c51da7f339a655e1d0c1999817a0bc4bf7520"} Jul 24 18:20:06.162585 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:06.162525 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podStartSLOduration=1.162504522 podStartE2EDuration="1.162504522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:20:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:20:06.162166434 +0000 UTC m=+2222.363398839" watchObservedRunningTime="2026-07-24 18:20:06.162504522 +0000 UTC m=+2222.363736912" Jul 24 18:20:06.681794 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:06.681751 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:06.696994 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:06.696953 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:07.147016 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:07.146984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:07.148061 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:07.148037 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:09.145630 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.145594 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:20:09.148188 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.148166 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.151041 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.151021 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-f556c1\"" Jul 24 18:20:09.162762 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.162738 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:20:09.231591 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.231558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.231775 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.231638 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-str4b\" (UniqueName: \"kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.332939 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.332902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.333132 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.332979 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-str4b\" (UniqueName: \"kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.333741 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.333720 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.342037 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.342013 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-str4b\" (UniqueName: \"kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b\") pod \"payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.460045 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.459950 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:09.652502 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:09.652479 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:20:09.654847 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:20:09.654806 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30694b1b_61f4_4e06_8910_2def843b9114.slice/crio-77631bdefbdc2f67e6f53e88b3807f9dbef74fcde76a994c3d9e166137275a58 WatchSource:0}: Error finding container 77631bdefbdc2f67e6f53e88b3807f9dbef74fcde76a994c3d9e166137275a58: Status 404 returned error can't find the container with id 77631bdefbdc2f67e6f53e88b3807f9dbef74fcde76a994c3d9e166137275a58 Jul 24 18:20:10.166353 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:10.166317 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" event={"ID":"30694b1b-61f4-4e06-8910-2def843b9114","Type":"ContainerStarted","Data":"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b"} Jul 24 18:20:10.166733 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:10.166388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" event={"ID":"30694b1b-61f4-4e06-8910-2def843b9114","Type":"ContainerStarted","Data":"77631bdefbdc2f67e6f53e88b3807f9dbef74fcde76a994c3d9e166137275a58"} Jul 24 18:20:10.166733 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:10.166416 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:10.190174 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:10.190108 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" podStartSLOduration=1.190082829 podStartE2EDuration="1.190082829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:20:10.187238867 +0000 UTC m=+2226.388471254" watchObservedRunningTime="2026-07-24 18:20:10.190082829 +0000 UTC m=+2226.391315219" Jul 24 18:20:13.190826 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.189416 2566 generic.go:356] "Generic (PLEG): container finished" podID="6487aa76-6978-46bc-bcb9-0c6268377206" containerID="b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591" exitCode=0 Jul 24 18:20:13.190826 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.189548 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerDied","Data":"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591"} Jul 24 18:20:13.190826 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.190094 2566 scope.go:117] "RemoveContainer" containerID="b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591" Jul 24 18:20:13.193309 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.192856 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:13.193309 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.192964 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:13.193309 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:13.193053 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:14.095780 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:14.095733 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:14.200623 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:14.200584 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerStarted","Data":"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832"} Jul 24 18:20:15.079963 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:15.079822 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:15.080201 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:15.079902 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 18:20:15.080201 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:15.080063 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 18:20:16.085387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:16.081841 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 18:20:16.085387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:16.081912 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 18:20:17.078028 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:17.077970 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 18:20:17.078265 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:17.078052 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 18:20:18.078476 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:18.078433 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 18:20:18.079001 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:18.078514 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 18:20:19.078089 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:19.077984 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 18:20:19.078837 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:19.078355 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 18:20:20.083069 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:20.083038 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:20.083534 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:20.083469 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:20.084348 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:20.084322 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:20:21.175570 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:21.175540 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:20:22.148527 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.148492 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:22.148757 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.148551 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:22.259565 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.259527 2566 generic.go:356] "Generic (PLEG): container finished" podID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerID="4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee" exitCode=0 Jul 24 18:20:22.260047 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.259607 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerDied","Data":"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee"} Jul 24 18:20:22.260131 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.260113 2566 scope.go:117] "RemoveContainer" containerID="4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee" Jul 24 18:20:22.261217 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.261191 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:22.261277 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.261264 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:22.261318 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:22.261297 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 24 18:20:23.267423 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:23.267371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerStarted","Data":"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e"} Jul 24 18:20:23.680173 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:23.680137 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:23.680484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:23.680452 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:23.680681 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:23.680651 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:24.679512 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:24.679468 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:24.680548 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:24.680103 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:25.679647 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:25.679606 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:25.680163 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:25.679698 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:26.680003 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:26.679910 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:26.680617 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:26.680585 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:27.679325 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:27.679279 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:27.679699 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:27.679352 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:28.679849 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:28.679815 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 24 18:20:28.680265 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:28.679867 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 24 18:20:29.683947 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:29.683911 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:29.684378 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:29.684180 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:20:29.684946 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:20:29.684928 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:25:17.115929 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.115893 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7539-139713846703664"] Jul 24 18:25:17.119405 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.119387 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:17.123548 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.123527 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ksjv4\"" Jul 24 18:25:17.142590 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.142561 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7539-139713846703664"] Jul 24 18:25:17.265650 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.265615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvsn7\" (UniqueName: \"kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7\") pod \"test-curl-7539-139713846703664\" (UID: \"c749178f-f80d-4096-aacc-6df0174d0c5e\") " pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:17.367086 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.366995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvsn7\" (UniqueName: \"kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7\") pod \"test-curl-7539-139713846703664\" (UID: \"c749178f-f80d-4096-aacc-6df0174d0c5e\") " pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:17.378125 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.378090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvsn7\" (UniqueName: \"kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7\") pod \"test-curl-7539-139713846703664\" (UID: \"c749178f-f80d-4096-aacc-6df0174d0c5e\") " pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:17.428667 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.428628 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:17.575221 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.575194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7539-139713846703664"] Jul 24 18:25:17.576849 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:25:17.576822 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc749178f_f80d_4096_aacc_6df0174d0c5e.slice/crio-7774f2fb3bf7bf4c48c574abfc394bb5b003b7ac61d4ddba070838b78396a555 WatchSource:0}: Error finding container 7774f2fb3bf7bf4c48c574abfc394bb5b003b7ac61d4ddba070838b78396a555: Status 404 returned error can't find the container with id 7774f2fb3bf7bf4c48c574abfc394bb5b003b7ac61d4ddba070838b78396a555 Jul 24 18:25:17.578080 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.578063 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 18:25:17.891644 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:17.891603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7539-139713846703664" event={"ID":"c749178f-f80d-4096-aacc-6df0174d0c5e","Type":"ContainerStarted","Data":"7774f2fb3bf7bf4c48c574abfc394bb5b003b7ac61d4ddba070838b78396a555"} Jul 24 18:25:18.898228 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:18.898191 2566 generic.go:356] "Generic (PLEG): container finished" podID="c749178f-f80d-4096-aacc-6df0174d0c5e" containerID="34f81123c23286eb4f8983a0f0fb705e5feed509a66e3e839373dbb079f6bc52" exitCode=0 Jul 24 18:25:18.898698 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:18.898270 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7539-139713846703664" event={"ID":"c749178f-f80d-4096-aacc-6df0174d0c5e","Type":"ContainerDied","Data":"34f81123c23286eb4f8983a0f0fb705e5feed509a66e3e839373dbb079f6bc52"} Jul 24 18:25:20.036182 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.036154 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:20.072843 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.072793 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7539-139713846703664_c749178f-f80d-4096-aacc-6df0174d0c5e/test-curl-7539-139713846703664/0.log" Jul 24 18:25:20.100455 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.100418 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7539-139713846703664"] Jul 24 18:25:20.105177 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.105149 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7539-139713846703664"] Jul 24 18:25:20.199220 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.199117 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvsn7\" (UniqueName: \"kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7\") pod \"c749178f-f80d-4096-aacc-6df0174d0c5e\" (UID: \"c749178f-f80d-4096-aacc-6df0174d0c5e\") " Jul 24 18:25:20.201356 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.201324 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7" (OuterVolumeSpecName: "kube-api-access-nvsn7") pod "c749178f-f80d-4096-aacc-6df0174d0c5e" (UID: "c749178f-f80d-4096-aacc-6df0174d0c5e"). InnerVolumeSpecName "kube-api-access-nvsn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:25:20.233069 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.233030 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7539-139713846539824"] Jul 24 18:25:20.233543 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.233523 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c749178f-f80d-4096-aacc-6df0174d0c5e" containerName="test-curl-7539-139713846703664" Jul 24 18:25:20.233646 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.233545 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c749178f-f80d-4096-aacc-6df0174d0c5e" containerName="test-curl-7539-139713846703664" Jul 24 18:25:20.233646 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.233642 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c749178f-f80d-4096-aacc-6df0174d0c5e" containerName="test-curl-7539-139713846703664" Jul 24 18:25:20.236883 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.236863 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:20.241238 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.241210 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7539-139713846539824"] Jul 24 18:25:20.250727 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.250702 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c749178f-f80d-4096-aacc-6df0174d0c5e" path="/var/lib/kubelet/pods/c749178f-f80d-4096-aacc-6df0174d0c5e/volumes" Jul 24 18:25:20.299934 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.299901 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvsn7\" (UniqueName: \"kubernetes.io/projected/c749178f-f80d-4096-aacc-6df0174d0c5e-kube-api-access-nvsn7\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:25:20.401348 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.401303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s46s\" (UniqueName: \"kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s\") pod \"test-curl-7539-139713846539824\" (UID: \"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf\") " pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:20.502641 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.502556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s46s\" (UniqueName: \"kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s\") pod \"test-curl-7539-139713846539824\" (UID: \"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf\") " pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:20.511459 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.511433 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s46s\" (UniqueName: \"kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s\") pod \"test-curl-7539-139713846539824\" (UID: \"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf\") " pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:20.548509 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.548477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:20.683166 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.683134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7539-139713846539824"] Jul 24 18:25:20.684332 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:25:20.684307 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5e6f9cc_5f7c_4ea9_ad55_78338bd12ecf.slice/crio-1925e029e0fc7621c5d97f6c37b25764495bccca4dd72fac6ba67994ed2b194c WatchSource:0}: Error finding container 1925e029e0fc7621c5d97f6c37b25764495bccca4dd72fac6ba67994ed2b194c: Status 404 returned error can't find the container with id 1925e029e0fc7621c5d97f6c37b25764495bccca4dd72fac6ba67994ed2b194c Jul 24 18:25:20.910122 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.910092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7539-139713846539824" event={"ID":"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf","Type":"ContainerStarted","Data":"1925e029e0fc7621c5d97f6c37b25764495bccca4dd72fac6ba67994ed2b194c"} Jul 24 18:25:20.911232 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.911212 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846703664" Jul 24 18:25:20.911341 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:20.911211 2566 scope.go:117] "RemoveContainer" containerID="34f81123c23286eb4f8983a0f0fb705e5feed509a66e3e839373dbb079f6bc52" Jul 24 18:25:21.918409 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:21.918352 2566 generic.go:356] "Generic (PLEG): container finished" podID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" containerID="e566f07d7d6bcdd309ed5e05d5fdabcdb89df9aeb798f1b3c2cf166377dc4942" exitCode=0 Jul 24 18:25:21.918860 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:21.918451 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7539-139713846539824" event={"ID":"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf","Type":"ContainerDied","Data":"e566f07d7d6bcdd309ed5e05d5fdabcdb89df9aeb798f1b3c2cf166377dc4942"} Jul 24 18:25:23.062064 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.062037 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:23.082867 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.082838 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7539-139713846539824_f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf/test-curl-7539-139713846539824/0.log" Jul 24 18:25:23.114045 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.111746 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7539-139713846539824"] Jul 24 18:25:23.116828 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.116803 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7539-139713846539824"] Jul 24 18:25:23.231114 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.231029 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s46s\" (UniqueName: \"kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s\") pod \"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf\" (UID: \"f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf\") " Jul 24 18:25:23.233073 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.233042 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s" (OuterVolumeSpecName: "kube-api-access-6s46s") pod "f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" (UID: "f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf"). InnerVolumeSpecName "kube-api-access-6s46s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:25:23.332207 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.332160 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s46s\" (UniqueName: \"kubernetes.io/projected/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf-kube-api-access-6s46s\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:25:23.934780 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.934748 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1925e029e0fc7621c5d97f6c37b25764495bccca4dd72fac6ba67994ed2b194c" Jul 24 18:25:23.934949 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:23.934793 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:23.944836 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:25:23.944805 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7539-139713846539824\" is forbidden: User \"system:node:ip-10-0-132-71.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-71.ec2.internal' and this object" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:24.250004 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:25:24.249903 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7539-139713846539824\" is forbidden: User \"system:node:ip-10-0-132-71.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-71.ec2.internal' and this object" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" pod="opendatahub/test-curl-7539-139713846539824" Jul 24 18:25:24.251149 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:24.251128 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" path="/var/lib/kubelet/pods/f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf/volumes" Jul 24 18:25:40.556944 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.556903 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:25:40.557482 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.557462 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" containerName="test-curl-7539-139713846539824" Jul 24 18:25:40.557565 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.557484 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" containerName="test-curl-7539-139713846539824" Jul 24 18:25:40.557621 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.557610 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e6f9cc-5f7c-4ea9-ad55-78338bd12ecf" containerName="test-curl-7539-139713846539824" Jul 24 18:25:40.562967 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.562944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.566179 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.566157 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d91e94-dockercfg-4hlll\"" Jul 24 18:25:40.566773 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.566753 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d91e94\"" Jul 24 18:25:40.574739 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.574716 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:25:40.584108 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.584084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5rrp\" (UniqueName: \"kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.584211 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.584180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.685476 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.685432 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5rrp\" (UniqueName: \"kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.685695 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.685563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.686229 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.686210 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.694189 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.694165 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5rrp\" (UniqueName: \"kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp\") pod \"payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:40.873711 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:40.873672 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:41.017498 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:41.017466 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:25:41.018497 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:25:41.018472 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898b0ed6_dd5b_4ade_992e_8682c42b4240.slice/crio-45f089858e4f45da812dddeabc3a66e6e44cbb6734123284f8c6648ad4736fb9 WatchSource:0}: Error finding container 45f089858e4f45da812dddeabc3a66e6e44cbb6734123284f8c6648ad4736fb9: Status 404 returned error can't find the container with id 45f089858e4f45da812dddeabc3a66e6e44cbb6734123284f8c6648ad4736fb9 Jul 24 18:25:41.027072 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:41.027049 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" event={"ID":"898b0ed6-dd5b-4ade-992e-8682c42b4240","Type":"ContainerStarted","Data":"45f089858e4f45da812dddeabc3a66e6e44cbb6734123284f8c6648ad4736fb9"} Jul 24 18:25:42.034839 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:42.034806 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" event={"ID":"898b0ed6-dd5b-4ade-992e-8682c42b4240","Type":"ContainerStarted","Data":"c9e28502ef27930e3ba7643e6eed77d5fcc7bc4f3ccdfb8c904593a1858ac9e0"} Jul 24 18:25:42.035229 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:42.034937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:42.052389 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:42.052326 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" podStartSLOduration=2.052310867 podStartE2EDuration="2.052310867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:25:42.050526871 +0000 UTC m=+2558.251759259" watchObservedRunningTime="2026-07-24 18:25:42.052310867 +0000 UTC m=+2558.253543255" Jul 24 18:25:53.043143 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:53.043111 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:25:59.303781 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.303741 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:25:59.307487 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.307468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.310298 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.310272 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-651f67-dockercfg-swwbb\"" Jul 24 18:25:59.310442 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.310331 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-651f67\"" Jul 24 18:25:59.317805 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.317774 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:25:59.357273 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.357230 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.357512 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.357310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvp9x\" (UniqueName: \"kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.458195 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.458160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.458413 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.458221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvp9x\" (UniqueName: \"kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.458950 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.458923 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.477481 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.477451 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvp9x\" (UniqueName: \"kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x\") pod \"payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.619477 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.619439 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:25:59.758466 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:25:59.758434 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:25:59.759701 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:25:59.759673 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4897d0_241e_41fd_8d24_85575966e8fb.slice/crio-1f18a700d6235a8350c898a1e062f301350e2bfe0ee4f939e3f0e7b27e32f90e WatchSource:0}: Error finding container 1f18a700d6235a8350c898a1e062f301350e2bfe0ee4f939e3f0e7b27e32f90e: Status 404 returned error can't find the container with id 1f18a700d6235a8350c898a1e062f301350e2bfe0ee4f939e3f0e7b27e32f90e Jul 24 18:26:00.134736 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:00.134699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" event={"ID":"fa4897d0-241e-41fd-8d24-85575966e8fb","Type":"ContainerStarted","Data":"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5"} Jul 24 18:26:00.134908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:00.134762 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:26:00.134908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:00.134788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" event={"ID":"fa4897d0-241e-41fd-8d24-85575966e8fb","Type":"ContainerStarted","Data":"1f18a700d6235a8350c898a1e062f301350e2bfe0ee4f939e3f0e7b27e32f90e"} Jul 24 18:26:00.158191 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:00.158131 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" podStartSLOduration=1.158116653 podStartE2EDuration="1.158116653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:26:00.152042707 +0000 UTC m=+2576.353275099" watchObservedRunningTime="2026-07-24 18:26:00.158116653 +0000 UTC m=+2576.359349042" Jul 24 18:26:11.143417 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:11.143386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:26:20.362647 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:20.362580 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg_fa4897d0-241e-41fd-8d24-85575966e8fb/payload-processing/0.log" Jul 24 18:26:56.916162 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:56.916129 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8_898b0ed6-dd5b-4ade-992e-8682c42b4240/payload-processing/0.log" Jul 24 18:26:57.178919 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:26:57.178834 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg_fa4897d0-241e-41fd-8d24-85575966e8fb/payload-processing/0.log" Jul 24 18:27:02.539884 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.539849 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:02.543305 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.543286 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.548267 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.548244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-1fd14b-dockercfg-gvd2k\"" Jul 24 18:27:02.549341 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.549323 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-1fd14b\"" Jul 24 18:27:02.549656 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.549634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.549711 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.549678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqwjs\" (UniqueName: \"kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.579906 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.579876 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:02.583555 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.583530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.588593 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.588563 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:02.619929 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.619905 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:02.650831 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.650798 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9v9k\" (UniqueName: \"kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.651032 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.650846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.651032 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.650868 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.651032 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.650888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqwjs\" (UniqueName: \"kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.651688 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.651668 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.674314 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.674284 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqwjs\" (UniqueName: \"kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs\") pod \"payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.751525 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.751476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9v9k\" (UniqueName: \"kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.751525 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.751536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.752332 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.752311 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.787203 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.787166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9v9k\" (UniqueName: \"kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k\") pod \"payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:02.854104 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.854068 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:02.895550 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:02.895510 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:03.039114 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.039088 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:03.039882 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:27:03.039859 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ef7aa5_6c8f_46f4_9cdd_2f32251841e8.slice/crio-1d08097dad9ea5ca7dc54ff4786f3aa6b60b602f57d24921e2407a4a2abbcdde WatchSource:0}: Error finding container 1d08097dad9ea5ca7dc54ff4786f3aa6b60b602f57d24921e2407a4a2abbcdde: Status 404 returned error can't find the container with id 1d08097dad9ea5ca7dc54ff4786f3aa6b60b602f57d24921e2407a4a2abbcdde Jul 24 18:27:03.272144 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.272119 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:03.274098 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:27:03.274065 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac42ca61_2614_4d86_b5b4_3498ac715c26.slice/crio-940afe2c329eeb9d28d8d3c54cd47df9060f62cc94d7b6a49a9447c40c202b1c WatchSource:0}: Error finding container 940afe2c329eeb9d28d8d3c54cd47df9060f62cc94d7b6a49a9447c40c202b1c: Status 404 returned error can't find the container with id 940afe2c329eeb9d28d8d3c54cd47df9060f62cc94d7b6a49a9447c40c202b1c Jul 24 18:27:03.496190 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.496157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" event={"ID":"ac42ca61-2614-4d86-b5b4-3498ac715c26","Type":"ContainerStarted","Data":"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093"} Jul 24 18:27:03.496414 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.496215 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" event={"ID":"ac42ca61-2614-4d86-b5b4-3498ac715c26","Type":"ContainerStarted","Data":"940afe2c329eeb9d28d8d3c54cd47df9060f62cc94d7b6a49a9447c40c202b1c"} Jul 24 18:27:03.496414 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.496267 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:03.497849 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.497824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" event={"ID":"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8","Type":"ContainerStarted","Data":"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187"} Jul 24 18:27:03.497984 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.497866 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:03.497984 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.497883 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" event={"ID":"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8","Type":"ContainerStarted","Data":"1d08097dad9ea5ca7dc54ff4786f3aa6b60b602f57d24921e2407a4a2abbcdde"} Jul 24 18:27:03.523168 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.523060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" podStartSLOduration=1.523042206 podStartE2EDuration="1.523042206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:03.521991231 +0000 UTC m=+2639.723223619" watchObservedRunningTime="2026-07-24 18:27:03.523042206 +0000 UTC m=+2639.724274576" Jul 24 18:27:03.543333 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:03.543275 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" podStartSLOduration=1.543258494 podStartE2EDuration="1.543258494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:03.540025118 +0000 UTC m=+2639.741257518" watchObservedRunningTime="2026-07-24 18:27:03.543258494 +0000 UTC m=+2639.744490883" Jul 24 18:27:14.510154 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:14.510123 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:14.510662 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:14.510194 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:27.115411 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.115371 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:27.115913 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.115629 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" podUID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" containerName="payload-processing" containerID="cri-o://deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187" gracePeriod=30 Jul 24 18:27:27.124330 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.124105 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:27.124444 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.124343 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" podUID="ac42ca61-2614-4d86-b5b4-3498ac715c26" containerName="payload-pre-processing" containerID="cri-o://26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093" gracePeriod=30 Jul 24 18:27:27.409656 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.409632 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:27.412447 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.412427 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:27.493858 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.493814 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqwjs\" (UniqueName: \"kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs\") pod \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " Jul 24 18:27:27.493858 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.493863 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume\") pod \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\" (UID: \"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8\") " Jul 24 18:27:27.494109 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.493915 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume\") pod \"ac42ca61-2614-4d86-b5b4-3498ac715c26\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " Jul 24 18:27:27.494109 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.493946 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9v9k\" (UniqueName: \"kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k\") pod \"ac42ca61-2614-4d86-b5b4-3498ac715c26\" (UID: \"ac42ca61-2614-4d86-b5b4-3498ac715c26\") " Jul 24 18:27:27.494338 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.494312 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" (UID: "70ef7aa5-6c8f-46f4-9cdd-2f32251841e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:27:27.494338 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.494324 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac42ca61-2614-4d86-b5b4-3498ac715c26" (UID: "ac42ca61-2614-4d86-b5b4-3498ac715c26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:27:27.496099 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.496069 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k" (OuterVolumeSpecName: "kube-api-access-v9v9k") pod "ac42ca61-2614-4d86-b5b4-3498ac715c26" (UID: "ac42ca61-2614-4d86-b5b4-3498ac715c26"). InnerVolumeSpecName "kube-api-access-v9v9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:27:27.496099 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.496084 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs" (OuterVolumeSpecName: "kube-api-access-wqwjs") pod "70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" (UID: "70ef7aa5-6c8f-46f4-9cdd-2f32251841e8"). InnerVolumeSpecName "kube-api-access-wqwjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:27:27.595649 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.595610 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac42ca61-2614-4d86-b5b4-3498ac715c26-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:27.595649 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.595642 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9v9k\" (UniqueName: \"kubernetes.io/projected/ac42ca61-2614-4d86-b5b4-3498ac715c26-kube-api-access-v9v9k\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:27.595649 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.595651 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqwjs\" (UniqueName: \"kubernetes.io/projected/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-kube-api-access-wqwjs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:27.595649 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.595660 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:27.640981 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.640889 2566 generic.go:356] "Generic (PLEG): container finished" podID="ac42ca61-2614-4d86-b5b4-3498ac715c26" containerID="26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093" exitCode=0 Jul 24 18:27:27.640981 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.640928 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" Jul 24 18:27:27.641219 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.640980 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" event={"ID":"ac42ca61-2614-4d86-b5b4-3498ac715c26","Type":"ContainerDied","Data":"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093"} Jul 24 18:27:27.641219 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.641029 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk" event={"ID":"ac42ca61-2614-4d86-b5b4-3498ac715c26","Type":"ContainerDied","Data":"940afe2c329eeb9d28d8d3c54cd47df9060f62cc94d7b6a49a9447c40c202b1c"} Jul 24 18:27:27.641219 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.641052 2566 scope.go:117] "RemoveContainer" containerID="26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093" Jul 24 18:27:27.642564 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.642538 2566 generic.go:356] "Generic (PLEG): container finished" podID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" containerID="deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187" exitCode=0 Jul 24 18:27:27.642683 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.642582 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" Jul 24 18:27:27.642683 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.642602 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" event={"ID":"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8","Type":"ContainerDied","Data":"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187"} Jul 24 18:27:27.642683 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.642657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk" event={"ID":"70ef7aa5-6c8f-46f4-9cdd-2f32251841e8","Type":"ContainerDied","Data":"1d08097dad9ea5ca7dc54ff4786f3aa6b60b602f57d24921e2407a4a2abbcdde"} Jul 24 18:27:27.651478 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.651444 2566 scope.go:117] "RemoveContainer" containerID="26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093" Jul 24 18:27:27.651785 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:27:27.651765 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093\": container with ID starting with 26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093 not found: ID does not exist" containerID="26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093" Jul 24 18:27:27.652014 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.651795 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093"} err="failed to get container status \"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093\": rpc error: code = NotFound desc = could not find container \"26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093\": container with ID starting with 26a6ac1c0ef8ccf0fb081a53002f044e2a913b4261bb97a91649a3f7d562c093 not found: ID does not exist" Jul 24 18:27:27.652014 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.651811 2566 scope.go:117] "RemoveContainer" containerID="deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187" Jul 24 18:27:27.660342 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.660322 2566 scope.go:117] "RemoveContainer" containerID="deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187" Jul 24 18:27:27.660637 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:27:27.660617 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187\": container with ID starting with deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187 not found: ID does not exist" containerID="deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187" Jul 24 18:27:27.660700 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.660648 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187"} err="failed to get container status \"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187\": rpc error: code = NotFound desc = could not find container \"deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187\": container with ID starting with deb83fd36878f23dd1f5c48d5ced6226396c95485a847f2b74459a6821c88187 not found: ID does not exist" Jul 24 18:27:27.664911 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.664882 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:27.669123 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.669101 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1fd14b-6d5b7dfff6-ll8jk"] Jul 24 18:27:27.680586 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.680555 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:27.682818 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:27.682797 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-1fd14b-5d5c4499d8-fhtbk"] Jul 24 18:27:28.249352 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:28.249317 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" path="/var/lib/kubelet/pods/70ef7aa5-6c8f-46f4-9cdd-2f32251841e8/volumes" Jul 24 18:27:28.249767 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:28.249718 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac42ca61-2614-4d86-b5b4-3498ac715c26" path="/var/lib/kubelet/pods/ac42ca61-2614-4d86-b5b4-3498ac715c26/volumes" Jul 24 18:27:57.579397 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.579285 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:27:57.579998 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.579556 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" podUID="898b0ed6-dd5b-4ade-992e-8682c42b4240" containerName="payload-processing" containerID="cri-o://c9e28502ef27930e3ba7643e6eed77d5fcc7bc4f3ccdfb8c904593a1858ac9e0" gracePeriod=30 Jul 24 18:27:57.817943 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.817914 2566 generic.go:356] "Generic (PLEG): container finished" podID="898b0ed6-dd5b-4ade-992e-8682c42b4240" containerID="c9e28502ef27930e3ba7643e6eed77d5fcc7bc4f3ccdfb8c904593a1858ac9e0" exitCode=0 Jul 24 18:27:57.818086 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.817982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" event={"ID":"898b0ed6-dd5b-4ade-992e-8682c42b4240","Type":"ContainerDied","Data":"c9e28502ef27930e3ba7643e6eed77d5fcc7bc4f3ccdfb8c904593a1858ac9e0"} Jul 24 18:27:57.835635 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.835564 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:27:57.887567 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.887528 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume\") pod \"898b0ed6-dd5b-4ade-992e-8682c42b4240\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " Jul 24 18:27:57.887759 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.887628 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5rrp\" (UniqueName: \"kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp\") pod \"898b0ed6-dd5b-4ade-992e-8682c42b4240\" (UID: \"898b0ed6-dd5b-4ade-992e-8682c42b4240\") " Jul 24 18:27:57.887924 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.887894 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "898b0ed6-dd5b-4ade-992e-8682c42b4240" (UID: "898b0ed6-dd5b-4ade-992e-8682c42b4240"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:27:57.889723 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.889694 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp" (OuterVolumeSpecName: "kube-api-access-q5rrp") pod "898b0ed6-dd5b-4ade-992e-8682c42b4240" (UID: "898b0ed6-dd5b-4ade-992e-8682c42b4240"). InnerVolumeSpecName "kube-api-access-q5rrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:27:57.988656 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.988620 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/898b0ed6-dd5b-4ade-992e-8682c42b4240-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:57.988656 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:57.988651 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5rrp\" (UniqueName: \"kubernetes.io/projected/898b0ed6-dd5b-4ade-992e-8682c42b4240-kube-api-access-q5rrp\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:27:58.825189 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:58.825152 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" Jul 24 18:27:58.825189 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:58.825177 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8" event={"ID":"898b0ed6-dd5b-4ade-992e-8682c42b4240","Type":"ContainerDied","Data":"45f089858e4f45da812dddeabc3a66e6e44cbb6734123284f8c6648ad4736fb9"} Jul 24 18:27:58.825739 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:58.825245 2566 scope.go:117] "RemoveContainer" containerID="c9e28502ef27930e3ba7643e6eed77d5fcc7bc4f3ccdfb8c904593a1858ac9e0" Jul 24 18:27:58.851256 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:58.851219 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:27:58.854214 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:27:58.854187 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d91e94-7d856f4694-hbnf8"] Jul 24 18:28:00.249085 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:28:00.249047 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="898b0ed6-dd5b-4ade-992e-8682c42b4240" path="/var/lib/kubelet/pods/898b0ed6-dd5b-4ade-992e-8682c42b4240/volumes" Jul 24 18:29:01.834059 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:01.834020 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:29:01.834944 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:01.834887 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" podUID="fa4897d0-241e-41fd-8d24-85575966e8fb" containerName="payload-processing" containerID="cri-o://ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5" gracePeriod=30 Jul 24 18:29:02.100917 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.100888 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:29:02.108408 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.108379 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvp9x\" (UniqueName: \"kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x\") pod \"fa4897d0-241e-41fd-8d24-85575966e8fb\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " Jul 24 18:29:02.108557 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.108512 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume\") pod \"fa4897d0-241e-41fd-8d24-85575966e8fb\" (UID: \"fa4897d0-241e-41fd-8d24-85575966e8fb\") " Jul 24 18:29:02.108878 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.108852 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa4897d0-241e-41fd-8d24-85575966e8fb" (UID: "fa4897d0-241e-41fd-8d24-85575966e8fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:29:02.110524 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.110501 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x" (OuterVolumeSpecName: "kube-api-access-jvp9x") pod "fa4897d0-241e-41fd-8d24-85575966e8fb" (UID: "fa4897d0-241e-41fd-8d24-85575966e8fb"). InnerVolumeSpecName "kube-api-access-jvp9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:02.180610 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.180572 2566 generic.go:356] "Generic (PLEG): container finished" podID="fa4897d0-241e-41fd-8d24-85575966e8fb" containerID="ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5" exitCode=0 Jul 24 18:29:02.180793 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.180621 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" Jul 24 18:29:02.180793 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.180620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" event={"ID":"fa4897d0-241e-41fd-8d24-85575966e8fb","Type":"ContainerDied","Data":"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5"} Jul 24 18:29:02.180793 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.180728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg" event={"ID":"fa4897d0-241e-41fd-8d24-85575966e8fb","Type":"ContainerDied","Data":"1f18a700d6235a8350c898a1e062f301350e2bfe0ee4f939e3f0e7b27e32f90e"} Jul 24 18:29:02.180793 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.180763 2566 scope.go:117] "RemoveContainer" containerID="ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5" Jul 24 18:29:02.190576 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.190552 2566 scope.go:117] "RemoveContainer" containerID="ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5" Jul 24 18:29:02.190892 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:29:02.190865 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5\": container with ID starting with ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5 not found: ID does not exist" containerID="ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5" Jul 24 18:29:02.190984 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.190903 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5"} err="failed to get container status \"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5\": rpc error: code = NotFound desc = could not find container \"ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5\": container with ID starting with ffef2d2f951ddf11c3cb8d348d08fe5fe71b753832a43d3512942a8de29fffd5 not found: ID does not exist" Jul 24 18:29:02.206966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.206936 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:29:02.210036 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.210012 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4897d0-241e-41fd-8d24-85575966e8fb-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:02.210115 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.210038 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvp9x\" (UniqueName: \"kubernetes.io/projected/fa4897d0-241e-41fd-8d24-85575966e8fb-kube-api-access-jvp9x\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:02.215755 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.215722 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651f67-f666fb8bb-m5whg"] Jul 24 18:29:02.249681 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:02.249648 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4897d0-241e-41fd-8d24-85575966e8fb" path="/var/lib/kubelet/pods/fa4897d0-241e-41fd-8d24-85575966e8fb/volumes" Jul 24 18:29:34.919651 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:34.919557 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:29:34.920237 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:34.919794 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" containerID="cri-o://2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832" gracePeriod=30 Jul 24 18:29:35.086301 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:35.086256 2566 patch_prober.go:28] interesting pod/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:29:35.086508 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:35.086315 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:29:40.179260 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.179232 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:29:40.266831 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266733 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.266831 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266773 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.266831 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266815 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59plw\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266850 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266886 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266919 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266944 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.266969 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267024 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket\") pod \"6487aa76-6978-46bc-bcb9-0c6268377206\" (UID: \"6487aa76-6978-46bc-bcb9-0c6268377206\") " Jul 24 18:29:40.267124 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267061 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data" (OuterVolumeSpecName: "istio-data") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:40.267526 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267306 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:40.267526 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267351 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:40.267526 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267427 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:29:40.267526 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267494 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:40.267720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267596 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.267720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267616 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.267720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267630 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6487aa76-6978-46bc-bcb9-0c6268377206-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.267720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267642 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.267720 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.267655 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.269213 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.269184 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw" (OuterVolumeSpecName: "kube-api-access-59plw") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "kube-api-access-59plw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:40.269347 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.269231 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token" (OuterVolumeSpecName: "istio-token") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:40.269347 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.269263 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:29:40.269347 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.269344 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6487aa76-6978-46bc-bcb9-0c6268377206" (UID: "6487aa76-6978-46bc-bcb9-0c6268377206"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:29:40.368609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.368557 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59plw\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-kube-api-access-59plw\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.368609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.368603 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6487aa76-6978-46bc-bcb9-0c6268377206-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.368609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.368613 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6487aa76-6978-46bc-bcb9-0c6268377206-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.368609 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.368623 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6487aa76-6978-46bc-bcb9-0c6268377206-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:40.403574 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.403536 2566 generic.go:356] "Generic (PLEG): container finished" podID="6487aa76-6978-46bc-bcb9-0c6268377206" containerID="2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832" exitCode=0 Jul 24 18:29:40.403750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.403594 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" Jul 24 18:29:40.403750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.403620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerDied","Data":"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832"} Jul 24 18:29:40.403750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.403673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r" event={"ID":"6487aa76-6978-46bc-bcb9-0c6268377206","Type":"ContainerDied","Data":"97e5d4095b22e83f0160fca906f1b2d2fa6f896c7f750346f789f1d15309d692"} Jul 24 18:29:40.403750 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.403694 2566 scope.go:117] "RemoveContainer" containerID="2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832" Jul 24 18:29:40.414167 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.414150 2566 scope.go:117] "RemoveContainer" containerID="b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591" Jul 24 18:29:40.422891 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.422870 2566 scope.go:117] "RemoveContainer" containerID="2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832" Jul 24 18:29:40.423132 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:29:40.423112 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832\": container with ID starting with 2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832 not found: ID does not exist" containerID="2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832" Jul 24 18:29:40.423200 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.423141 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832"} err="failed to get container status \"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832\": rpc error: code = NotFound desc = could not find container \"2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832\": container with ID starting with 2bc0e11f9c6ec4a319c505c65e3eabe01dce87f704c74de7ca8c03099c666832 not found: ID does not exist" Jul 24 18:29:40.423200 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.423159 2566 scope.go:117] "RemoveContainer" containerID="b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591" Jul 24 18:29:40.423414 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:29:40.423394 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591\": container with ID starting with b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591 not found: ID does not exist" containerID="b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591" Jul 24 18:29:40.423484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.423419 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591"} err="failed to get container status \"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591\": rpc error: code = NotFound desc = could not find container \"b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591\": container with ID starting with b3a4fd4b213f541a57ae5f3631c60227f6e8f0e7684ba1e72501804361af9591 not found: ID does not exist" Jul 24 18:29:40.428677 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.428651 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:29:40.433105 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:40.433083 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2542df-openshift-default-6b967b6764-f559r"] Jul 24 18:29:42.254604 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:42.254561 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" path="/var/lib/kubelet/pods/6487aa76-6978-46bc-bcb9-0c6268377206/volumes" Jul 24 18:29:45.713735 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:45.713694 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:29:45.714225 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:45.713978 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" podUID="30694b1b-61f4-4e06-8910-2def843b9114" containerName="payload-processing" containerID="cri-o://4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b" gracePeriod=30 Jul 24 18:29:45.974205 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:45.974140 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:29:46.023554 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.023527 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume\") pod \"30694b1b-61f4-4e06-8910-2def843b9114\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " Jul 24 18:29:46.023724 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.023592 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-str4b\" (UniqueName: \"kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b\") pod \"30694b1b-61f4-4e06-8910-2def843b9114\" (UID: \"30694b1b-61f4-4e06-8910-2def843b9114\") " Jul 24 18:29:46.023956 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.023935 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "30694b1b-61f4-4e06-8910-2def843b9114" (UID: "30694b1b-61f4-4e06-8910-2def843b9114"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:29:46.025667 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.025642 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b" (OuterVolumeSpecName: "kube-api-access-str4b") pod "30694b1b-61f4-4e06-8910-2def843b9114" (UID: "30694b1b-61f4-4e06-8910-2def843b9114"). InnerVolumeSpecName "kube-api-access-str4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:29:46.125331 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.125280 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-str4b\" (UniqueName: \"kubernetes.io/projected/30694b1b-61f4-4e06-8910-2def843b9114-kube-api-access-str4b\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:46.125331 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.125325 2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30694b1b-61f4-4e06-8910-2def843b9114-plugins-config-volume\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:29:46.437758 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.437720 2566 generic.go:356] "Generic (PLEG): container finished" podID="30694b1b-61f4-4e06-8910-2def843b9114" containerID="4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b" exitCode=0 Jul 24 18:29:46.437938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.437782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" Jul 24 18:29:46.437938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.437793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" event={"ID":"30694b1b-61f4-4e06-8910-2def843b9114","Type":"ContainerDied","Data":"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b"} Jul 24 18:29:46.437938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.437860 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv" event={"ID":"30694b1b-61f4-4e06-8910-2def843b9114","Type":"ContainerDied","Data":"77631bdefbdc2f67e6f53e88b3807f9dbef74fcde76a994c3d9e166137275a58"} Jul 24 18:29:46.437938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.437880 2566 scope.go:117] "RemoveContainer" containerID="4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b" Jul 24 18:29:46.447126 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.447092 2566 scope.go:117] "RemoveContainer" containerID="4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b" Jul 24 18:29:46.447433 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:29:46.447382 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b\": container with ID starting with 4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b not found: ID does not exist" containerID="4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b" Jul 24 18:29:46.447433 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.447412 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b"} err="failed to get container status \"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b\": rpc error: code = NotFound desc = could not find container \"4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b\": container with ID starting with 4e6bbbfc8aae43f0d6101c1ff7af7f84f779031b15f6a0161ec5ca0827557d0b not found: ID does not exist" Jul 24 18:29:46.458562 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.458531 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:29:46.463181 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:46.463152 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f556c1-6b64d578b-f6wmv"] Jul 24 18:29:48.248842 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:48.248810 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30694b1b-61f4-4e06-8910-2def843b9114" path="/var/lib/kubelet/pods/30694b1b-61f4-4e06-8910-2def843b9114/volumes" Jul 24 18:29:56.928175 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:56.928138 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:29:56.928693 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:56.928331 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" containerID="cri-o://45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e" gracePeriod=30 Jul 24 18:29:59.686921 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:59.686879 2566 patch_prober.go:28] interesting pod/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 18:29:59.687297 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:29:59.686954 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 18:30:02.178872 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.178844 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:30:02.274716 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274609 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274716 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274677 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274733 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274756 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274784 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-825t7\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274808 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274855 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274912 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.274966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.274955 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert\") pod \"1bf385ed-801e-4124-9343-1496a6b0bf24\" (UID: \"1bf385ed-801e-4124-9343-1496a6b0bf24\") " Jul 24 18:30:02.275313 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:30:02.275313 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275073 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data" (OuterVolumeSpecName: "istio-data") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:30:02.275313 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275144 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:30:02.275519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275343 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:30:02.275519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275350 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-certs\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.275519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275423 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-data\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.275519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275441 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-credential-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.275519 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.275464 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 18:30:02.277017 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.276983 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 18:30:02.277017 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.276993 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7" (OuterVolumeSpecName: "kube-api-access-825t7") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "kube-api-access-825t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:30:02.277253 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.277230 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 18:30:02.277380 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.277317 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token" (OuterVolumeSpecName: "istio-token") pod "1bf385ed-801e-4124-9343-1496a6b0bf24" (UID: "1bf385ed-801e-4124-9343-1496a6b0bf24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 18:30:02.376283 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376240 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-workload-socket\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.376283 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376278 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf385ed-801e-4124-9343-1496a6b0bf24-istiod-ca-cert\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.376283 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376293 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-envoy\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.376588 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376306 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-podinfo\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.376588 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376319 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-825t7\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-kube-api-access-825t7\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.376588 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.376330 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf385ed-801e-4124-9343-1496a6b0bf24-istio-token\") on node \"ip-10-0-132-71.ec2.internal\" DevicePath \"\"" Jul 24 18:30:02.526385 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.526273 2566 generic.go:356] "Generic (PLEG): container finished" podID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerID="45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e" exitCode=0 Jul 24 18:30:02.526385 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.526317 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" Jul 24 18:30:02.526385 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.526353 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerDied","Data":"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e"} Jul 24 18:30:02.526661 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.526430 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj" event={"ID":"1bf385ed-801e-4124-9343-1496a6b0bf24","Type":"ContainerDied","Data":"21563f195703ac7a1e78bd816b5c51da7f339a655e1d0c1999817a0bc4bf7520"} Jul 24 18:30:02.526661 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.526453 2566 scope.go:117] "RemoveContainer" containerID="45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e" Jul 24 18:30:02.536394 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.536369 2566 scope.go:117] "RemoveContainer" containerID="4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee" Jul 24 18:30:02.544945 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.544922 2566 scope.go:117] "RemoveContainer" containerID="45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e" Jul 24 18:30:02.545254 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:30:02.545231 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e\": container with ID starting with 45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e not found: ID does not exist" containerID="45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e" Jul 24 18:30:02.545335 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.545269 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e"} err="failed to get container status \"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e\": rpc error: code = NotFound desc = could not find container \"45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e\": container with ID starting with 45ab7c73fc63404ecb3544414809d43b7e4194b7186f8bbb1bf1bf0e8ad7252e not found: ID does not exist" Jul 24 18:30:02.545335 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.545297 2566 scope.go:117] "RemoveContainer" containerID="4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee" Jul 24 18:30:02.545586 ip-10-0-132-71 kubenswrapper[2566]: E0724 18:30:02.545566 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee\": container with ID starting with 4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee not found: ID does not exist" containerID="4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee" Jul 24 18:30:02.545633 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.545592 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee"} err="failed to get container status \"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee\": rpc error: code = NotFound desc = could not find container \"4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee\": container with ID starting with 4a313d8f8ab5282665d176bfbd40c8566dc613d31f9778f32736d43b630eadee not found: ID does not exist" Jul 24 18:30:02.552966 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.552928 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:30:02.555209 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:02.555189 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f556c1-openshift-default-68565d799b-dgkdj"] Jul 24 18:30:03.309403 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:03.309373 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l4szd_800f4d5e-f2ff-4f6c-b6cd-f449dd720137/server/0.log" Jul 24 18:30:03.776507 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:03.776476 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf4bd545-l2s9p_7c020c2e-014d-4a3c-be20-5681527c383c/copy-manifests/0.log" Jul 24 18:30:03.789591 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:03.789565 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf4bd545-l2s9p_7c020c2e-014d-4a3c-be20-5681527c383c/manager/0.log" Jul 24 18:30:04.235390 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:04.235344 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748600-m2ngc_8302a30a-b70f-49c0-b115-604dc9eec661/cleanup/0.log" Jul 24 18:30:04.250723 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:04.250689 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" path="/var/lib/kubelet/pods/1bf385ed-801e-4124-9343-1496a6b0bf24/volumes" Jul 24 18:30:05.644226 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:05.644196 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-4kczr_7639a455-08ce-4430-ae43-e3b73952f11e/kuadrant-console-plugin/0.log" Jul 24 18:30:06.682867 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:06.682834 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c646dc89-jpn2p_be8c0bad-e930-40b1-8cde-cf68fd838e0a/kube-auth-proxy/0.log" Jul 24 18:30:06.914284 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:06.914253 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tj4qf_2626cd9e-d17e-43fa-a7cf-3a18ce16848e/payload-pre-processing/0.log" Jul 24 18:30:07.468852 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:07.468819 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2tqhp_5a3d9c72-020f-401e-b03f-e61103d61f60/main/0.log" Jul 24 18:30:07.912234 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:07.912202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-47q5g_d7ebfd30-99fb-4e40-b6d1-d2b09e16a455/main/0.log" Jul 24 18:30:08.022676 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:08.022640 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-4hgkx_4821ddfb-f57a-4667-b34f-015b070b9bf3/main/0.log" Jul 24 18:30:15.814264 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:15.814223 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qdzsl_d6e14a3c-14e7-472b-9452-5b8f01744623/global-pull-secret-syncer/0.log" Jul 24 18:30:15.968155 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:15.968123 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kddtr_2254c944-7260-42a3-ba48-3c2dfea4695a/konnectivity-agent/0.log" Jul 24 18:30:16.043526 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:16.043492 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-71.ec2.internal_96d0a8774c9d2bcc0171d5a7c66343bb/haproxy/0.log" Jul 24 18:30:21.267800 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:21.267772 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-4kczr_7639a455-08ce-4430-ae43-e3b73952f11e/kuadrant-console-plugin/0.log" Jul 24 18:30:23.236045 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.236019 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/alertmanager/0.log" Jul 24 18:30:23.280928 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.280904 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/config-reloader/0.log" Jul 24 18:30:23.322133 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.322054 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/kube-rbac-proxy-web/0.log" Jul 24 18:30:23.350664 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.350633 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/kube-rbac-proxy/0.log" Jul 24 18:30:23.386341 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.386314 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/kube-rbac-proxy-metric/0.log" Jul 24 18:30:23.413869 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.413840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/prom-label-proxy/0.log" Jul 24 18:30:23.437931 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.437903 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7d37fdec-e69a-44c9-912e-949d991e220d/init-config-reloader/0.log" Jul 24 18:30:23.519948 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.519915 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-w99rs_3eb63ec5-cbdb-40c9-944d-41fb7225e00f/kube-state-metrics/0.log" Jul 24 18:30:23.547135 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.547104 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-w99rs_3eb63ec5-cbdb-40c9-944d-41fb7225e00f/kube-rbac-proxy-main/0.log" Jul 24 18:30:23.571203 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.571169 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-w99rs_3eb63ec5-cbdb-40c9-944d-41fb7225e00f/kube-rbac-proxy-self/0.log" Jul 24 18:30:23.703685 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.703656 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68c8z_288a623b-c0b6-41f6-a147-fb70fb82bf71/node-exporter/0.log" Jul 24 18:30:23.738075 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.738045 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68c8z_288a623b-c0b6-41f6-a147-fb70fb82bf71/kube-rbac-proxy/0.log" Jul 24 18:30:23.770995 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:23.770970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68c8z_288a623b-c0b6-41f6-a147-fb70fb82bf71/init-textfile/0.log" Jul 24 18:30:24.011801 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:24.011727 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2fp4l_336afd0d-d237-415b-89ab-c8474111be70/kube-rbac-proxy-main/0.log" Jul 24 18:30:24.064515 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:24.064485 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2fp4l_336afd0d-d237-415b-89ab-c8474111be70/kube-rbac-proxy-self/0.log" Jul 24 18:30:24.106693 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:24.106662 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-2fp4l_336afd0d-d237-415b-89ab-c8474111be70/openshift-state-metrics/0.log" Jul 24 18:30:24.533524 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:24.533497 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-sjh77_a57443aa-74ee-4cd2-86f0-1f9a7de1d42a/prometheus-operator-admission-webhook/0.log" Jul 24 18:30:26.125434 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125397 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26"] Jul 24 18:30:26.125843 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125825 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125846 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125859 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125864 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125870 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac42ca61-2614-4d86-b5b4-3498ac715c26" containerName="payload-pre-processing" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125876 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac42ca61-2614-4d86-b5b4-3498ac715c26" containerName="payload-pre-processing" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125885 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30694b1b-61f4-4e06-8910-2def843b9114" containerName="payload-processing" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125891 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30694b1b-61f4-4e06-8910-2def843b9114" containerName="payload-processing" Jul 24 18:30:26.125901 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125900 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125906 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125915 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125920 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125929 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="898b0ed6-dd5b-4ade-992e-8682c42b4240" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125933 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="898b0ed6-dd5b-4ade-992e-8682c42b4240" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125944 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4897d0-241e-41fd-8d24-85575966e8fb" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.125949 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4897d0-241e-41fd-8d24-85575966e8fb" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126003 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ef7aa5-6c8f-46f4-9cdd-2f32251841e8" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126011 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac42ca61-2614-4d86-b5b4-3498ac715c26" containerName="payload-pre-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126017 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf385ed-801e-4124-9343-1496a6b0bf24" containerName="istio-proxy" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126024 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6487aa76-6978-46bc-bcb9-0c6268377206" containerName="istio-proxy" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126028 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4897d0-241e-41fd-8d24-85575966e8fb" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126035 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="30694b1b-61f4-4e06-8910-2def843b9114" containerName="payload-processing" Jul 24 18:30:26.126145 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.126040 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="898b0ed6-dd5b-4ade-992e-8682c42b4240" containerName="payload-processing" Jul 24 18:30:26.133116 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.130051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.134384 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.134284 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jqhx8\"/\"openshift-service-ca.crt\"" Jul 24 18:30:26.136237 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.136202 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jqhx8\"/\"kube-root-ca.crt\"" Jul 24 18:30:26.136392 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.136254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jqhx8\"/\"default-dockercfg-sqr6w\"" Jul 24 18:30:26.140235 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.140045 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26"] Jul 24 18:30:26.189292 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.189245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-proc\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.189292 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.189295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-podres\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.189551 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.189329 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9lzr\" (UniqueName: \"kubernetes.io/projected/9ee8dc19-8c83-4fa1-96e3-59e89a360629-kube-api-access-s9lzr\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.189551 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.189355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-sys\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.189551 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.189466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-lib-modules\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290094 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9lzr\" (UniqueName: \"kubernetes.io/projected/9ee8dc19-8c83-4fa1-96e3-59e89a360629-kube-api-access-s9lzr\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290094 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-sys\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-lib-modules\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-proc\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-sys\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-podres\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290296 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-proc\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-lib-modules\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.290387 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.290387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ee8dc19-8c83-4fa1-96e3-59e89a360629-podres\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.298961 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.298937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9lzr\" (UniqueName: \"kubernetes.io/projected/9ee8dc19-8c83-4fa1-96e3-59e89a360629-kube-api-access-s9lzr\") pod \"perf-node-gather-daemonset-lvj26\" (UID: \"9ee8dc19-8c83-4fa1-96e3-59e89a360629\") " pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.444020 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.443916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:26.601769 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.601727 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26"] Jul 24 18:30:26.604317 ip-10-0-132-71 kubenswrapper[2566]: W0724 18:30:26.604289 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9ee8dc19_8c83_4fa1_96e3_59e89a360629.slice/crio-e3a3b08d038bee88ee56ce704435abb6c17e5acbd7513a5f301c198f401c97d2 WatchSource:0}: Error finding container e3a3b08d038bee88ee56ce704435abb6c17e5acbd7513a5f301c198f401c97d2: Status 404 returned error can't find the container with id e3a3b08d038bee88ee56ce704435abb6c17e5acbd7513a5f301c198f401c97d2 Jul 24 18:30:26.649982 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:26.649955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" event={"ID":"9ee8dc19-8c83-4fa1-96e3-59e89a360629","Type":"ContainerStarted","Data":"e3a3b08d038bee88ee56ce704435abb6c17e5acbd7513a5f301c198f401c97d2"} Jul 24 18:30:27.118484 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:27.118451 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65d6879f4f-qsbqw_06a182a9-b09b-40be-9506-983ed758ed97/console/0.log" Jul 24 18:30:27.657525 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:27.657495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" event={"ID":"9ee8dc19-8c83-4fa1-96e3-59e89a360629","Type":"ContainerStarted","Data":"c9c1ff0241386b849bf750cf5d6877a978d149252685b733d9504bd1e3e83f53"} Jul 24 18:30:27.657941 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:27.657599 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:27.674492 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:27.674438 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" podStartSLOduration=1.674422099 podStartE2EDuration="1.674422099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:30:27.673265229 +0000 UTC m=+2843.874497617" watchObservedRunningTime="2026-07-24 18:30:27.674422099 +0000 UTC m=+2843.875654487" Jul 24 18:30:28.927641 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:28.927612 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjbzn_6762dfe6-30a5-448c-afea-86fffc3ae2c7/dns/0.log" Jul 24 18:30:28.951045 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:28.951015 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjbzn_6762dfe6-30a5-448c-afea-86fffc3ae2c7/kube-rbac-proxy/0.log" Jul 24 18:30:29.024908 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:29.024875 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t9nvd_11364763-f913-455b-9687-aa0a8bc528bc/dns-node-resolver/0.log" Jul 24 18:30:29.637679 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:29.637647 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zhwnj_3ea17a03-d149-4a7c-9f03-47f288bf3f3c/node-ca/0.log" Jul 24 18:30:30.812103 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:30.812073 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c646dc89-jpn2p_be8c0bad-e930-40b1-8cde-cf68fd838e0a/kube-auth-proxy/0.log" Jul 24 18:30:30.893597 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:30.893560 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tj4qf_2626cd9e-d17e-43fa-a7cf-3a18ce16848e/payload-pre-processing/0.log" Jul 24 18:30:31.791324 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:31.791296 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8kf99_0d32485a-b722-4c1e-8024-22ca8fd832c3/serve-healthcheck-canary/0.log" Jul 24 18:30:32.530027 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:32.529996 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sl8lt_c5012c20-c068-4021-b3f9-f2ab33ffafa4/kube-rbac-proxy/0.log" Jul 24 18:30:32.561547 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:32.561519 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sl8lt_c5012c20-c068-4021-b3f9-f2ab33ffafa4/exporter/0.log" Jul 24 18:30:32.588319 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:32.588293 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sl8lt_c5012c20-c068-4021-b3f9-f2ab33ffafa4/extractor/0.log" Jul 24 18:30:33.673286 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:33.673256 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jqhx8/perf-node-gather-daemonset-lvj26" Jul 24 18:30:35.385938 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:35.385916 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l4szd_800f4d5e-f2ff-4f6c-b6cd-f449dd720137/server/0.log" Jul 24 18:30:35.678090 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:35.677996 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf4bd545-l2s9p_7c020c2e-014d-4a3c-be20-5681527c383c/manager/0.log" Jul 24 18:30:35.697451 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:35.697421 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf4bd545-l2s9p_7c020c2e-014d-4a3c-be20-5681527c383c/copy-manifests/0.log" Jul 24 18:30:36.787168 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:36.787143 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c646dc89-jpn2p_be8c0bad-e930-40b1-8cde-cf68fd838e0a/kube-auth-proxy/0.log" Jul 24 18:30:36.834353 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:36.834317 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tj4qf_2626cd9e-d17e-43fa-a7cf-3a18ce16848e/payload-pre-processing/0.log" Jul 24 18:30:37.010922 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:37.010887 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9f897544f-mk7mf_2602affc-9630-46ca-a0d1-cfef2fbf4b7f/manager/0.log" Jul 24 18:30:37.037500 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:37.037418 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cq9f2_e7770bc7-3eff-4bf8-85eb-eee11a50cc22/openshift-lws-operator/0.log" Jul 24 18:30:38.542751 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:38.542722 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56c646dc89-jpn2p_be8c0bad-e930-40b1-8cde-cf68fd838e0a/kube-auth-proxy/0.log" Jul 24 18:30:38.609232 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:38.609201 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-tj4qf_2626cd9e-d17e-43fa-a7cf-3a18ce16848e/payload-pre-processing/0.log" Jul 24 18:30:41.988728 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:41.988681 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-r9jtp_df2dd3b2-bfcb-4691-922d-8eacf64922f8/migrator/0.log" Jul 24 18:30:42.011030 ip-10-0-132-71 kubenswrapper[2566]: I0724 18:30:42.010995 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-r9jtp_df2dd3b2-bfcb-4691-922d-8eacf64922f8/graceful-termination/0.log"