Jul 23 17:04:38.534672 ip-10-0-136-29 systemd[1]: Starting Kubernetes Kubelet... Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.908926 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914332 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914352 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914373 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914382 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914388 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914392 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914396 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.023444 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914401 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:38.986646 ip-10-0-136-29 systemd[1]: Started Kubernetes Kubelet. Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914405 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914409 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914412 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914417 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914420 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914424 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914428 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914432 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914436 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914440 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914443 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914447 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914451 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914455 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914459 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914465 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914470 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914474 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.025532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914482 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.914487 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915837 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915843 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915846 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915849 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915853 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915855 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915859 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915861 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915864 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915867 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915871 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915874 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915876 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915879 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915882 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915885 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915889 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915891 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.026205 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915893 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915896 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915899 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915904 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915907 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915909 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915912 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915914 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915917 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915920 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915922 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915925 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915927 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915931 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915934 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915936 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915939 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915942 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915945 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915947 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.026881 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915950 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915952 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915955 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915959 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915961 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915964 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915967 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915969 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915972 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915976 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915979 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915981 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915984 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915986 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915989 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915991 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915994 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915996 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.915999 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916001 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.027682 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916005 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916010 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916013 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916016 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916019 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916022 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916025 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916027 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916030 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916032 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916035 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916037 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916040 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916814 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916823 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916825 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916828 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916832 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916834 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916837 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.028347 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916839 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916842 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916846 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916849 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916851 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916854 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916856 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916859 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916861 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916864 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916866 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916869 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916871 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916874 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916876 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916879 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916883 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916886 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916889 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916892 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.029136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916895 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916897 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916900 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916902 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916905 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916907 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916910 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916912 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916914 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916917 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916919 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916922 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916924 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916927 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916929 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916932 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916935 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916937 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916940 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.263906 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916942 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916946 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916950 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916953 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916955 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916958 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916960 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916966 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916969 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916972 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916975 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916978 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916980 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916983 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916985 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916987 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916990 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916992 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916995 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.281289 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.916997 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917000 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917002 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917004 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917008 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917012 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917016 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917019 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917021 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917024 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917027 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917029 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917033 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917036 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917038 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917041 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917043 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917046 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917048 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.281944 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917052 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917055 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917057 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917060 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917063 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917066 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917068 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917071 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917073 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917076 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917078 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917080 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917083 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917085 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917087 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917481 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917486 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917489 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917492 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917494 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.282568 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917497 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917500 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917503 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917518 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917520 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917525 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917527 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917530 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917533 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917535 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917538 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917540 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917543 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917546 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917548 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917551 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917554 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917556 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917559 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917561 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.283140 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917563 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917566 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917569 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917571 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917573 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917576 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917578 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917581 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917583 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917586 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917588 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917591 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917593 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917596 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917598 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917602 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917607 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917612 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917615 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.283761 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917618 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917621 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917623 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917626 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917629 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917631 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917634 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917636 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917639 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917641 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917644 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917646 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917650 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917654 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917656 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917659 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917662 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917664 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917667 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.284309 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917669 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917671 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917674 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917678 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917681 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917683 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917686 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917689 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917691 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917695 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917698 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917702 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917705 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917708 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917710 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917713 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917715 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917718 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917720 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917723 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.284932 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917725 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917728 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917730 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917733 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917735 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917737 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917740 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917742 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917745 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917747 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917750 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917752 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917755 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917757 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917760 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.917762 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917848 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917867 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917876 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917882 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917894 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 17:04:39.285475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917900 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917905 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917911 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917915 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917919 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917922 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917925 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917928 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917931 2584 flags.go:64] FLAG: --cgroup-root="" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917934 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917937 2584 flags.go:64] FLAG: --client-ca-file="" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917940 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917943 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917948 2584 flags.go:64] FLAG: --cluster-domain="" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917951 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917954 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917957 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917960 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917964 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917967 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917970 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917974 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917976 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917979 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 17:04:39.286057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917982 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917985 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917988 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917993 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917996 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.917999 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918001 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918004 2584 flags.go:64] FLAG: --enable-server="true" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918009 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918014 2584 flags.go:64] FLAG: --event-burst="100" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918018 2584 flags.go:64] FLAG: --event-qps="50" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918022 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918028 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918033 2584 flags.go:64] FLAG: --eviction-hard="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918037 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918040 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918043 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918046 2584 flags.go:64] FLAG: --eviction-soft="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918049 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918052 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918055 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918058 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918060 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918064 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918066 2584 flags.go:64] FLAG: --feature-gates="" Jul 23 17:04:39.286711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918075 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918078 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918081 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918084 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918087 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918090 2584 flags.go:64] FLAG: --help="false" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918093 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918096 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918099 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918102 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918105 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918108 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918111 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918114 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918117 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918121 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918124 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918128 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918131 2584 flags.go:64] FLAG: --kube-reserved="" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918134 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918138 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918141 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918144 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918147 2584 flags.go:64] FLAG: --lock-file="" Jul 23 17:04:39.287424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918150 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918152 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918155 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918161 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918163 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918166 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918169 2584 flags.go:64] FLAG: --logging-format="text" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918172 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918175 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918178 2584 flags.go:64] FLAG: --manifest-url="" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918180 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918185 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918188 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918192 2584 flags.go:64] FLAG: --max-pods="110" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918195 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918198 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918201 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918204 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918207 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918209 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918212 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918220 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918222 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918227 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 17:04:39.288049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918230 2584 flags.go:64] FLAG: --pod-cidr="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918234 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918240 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918244 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918247 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918250 2584 flags.go:64] FLAG: --port="10250" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918253 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918256 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c1858a06e032707e" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918259 2584 flags.go:64] FLAG: --qos-reserved="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918262 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918265 2584 flags.go:64] FLAG: --register-node="true" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918268 2584 flags.go:64] FLAG: --register-with-taints="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918272 2584 flags.go:64] FLAG: --registry-burst="10" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918275 2584 flags.go:64] FLAG: --registry-qps="5" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918277 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918280 2584 flags.go:64] FLAG: --reserved-memory="" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918284 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918287 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918290 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918293 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918295 2584 flags.go:64] FLAG: --runonce="false" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918298 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918301 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918304 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918307 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 17:04:39.288809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918310 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918313 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918316 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918319 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918322 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918325 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918327 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918332 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918336 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918338 2584 flags.go:64] FLAG: --system-cgroups="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918341 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918346 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918349 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918353 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918372 2584 flags.go:64] FLAG: --tls-min-version="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918375 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918378 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918381 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918384 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918387 2584 flags.go:64] FLAG: --v="2" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918391 2584 flags.go:64] FLAG: --version="false" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918395 2584 flags.go:64] FLAG: --vmodule="" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918398 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.918401 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 17:04:39.289753 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918533 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918537 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918540 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918543 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918545 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918548 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918550 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918553 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918556 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918558 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918562 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918565 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918568 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918571 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918573 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918577 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918581 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918584 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918586 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918589 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.290439 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918592 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918594 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918597 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918600 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918602 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918605 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918607 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918610 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918612 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918615 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918619 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918621 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918623 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918626 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918629 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918631 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918634 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918636 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918639 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918641 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.525759 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918644 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918646 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918649 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918651 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918654 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918656 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918659 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918664 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918668 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918671 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918673 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918676 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918678 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918681 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918683 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918693 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918695 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918698 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918700 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.526786 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918703 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918706 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918708 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918711 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918714 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918716 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918719 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918721 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918724 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918726 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918730 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918734 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918737 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918740 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918743 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918746 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918748 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918751 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918754 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.527354 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918756 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918761 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918766 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918768 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918771 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918773 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918776 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918779 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918783 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918786 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918790 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918792 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918795 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918798 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918800 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918803 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918806 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918808 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918810 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.527970 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918813 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.918815 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.919374 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927138 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927158 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927180 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927186 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927243 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927248 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927252 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927255 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927258 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927261 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927263 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927266 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.528533 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927269 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927272 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927274 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927277 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927279 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927282 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927284 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927287 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927290 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927293 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927295 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927298 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927300 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927302 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927305 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927307 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927310 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927312 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927315 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927318 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.529045 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927321 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927324 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927326 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927329 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927331 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927336 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927339 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927342 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927348 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927351 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927354 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927373 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927377 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927379 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927382 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927385 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927387 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927390 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927393 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927396 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.529651 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927398 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927402 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927406 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927410 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927414 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927418 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927421 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927424 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927427 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927430 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927433 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927436 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927439 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927442 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927446 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927448 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927451 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927454 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927457 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.530261 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927460 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927464 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927466 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927469 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927472 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927475 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927477 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927480 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927483 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927485 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927487 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927490 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927493 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927496 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927499 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927502 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927504 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927507 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927510 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927512 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.530841 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927515 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927518 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927520 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927523 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927526 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927528 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927530 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927533 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927535 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927538 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927540 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927542 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927547 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927660 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927667 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 17:04:39.531479 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927670 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927673 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927675 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927678 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927681 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927684 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927687 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927689 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927692 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927694 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927697 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927699 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927702 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927705 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927708 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927710 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927713 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927715 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927717 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927720 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 17:04:39.531920 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927722 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927725 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927728 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927732 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927736 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927739 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927741 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927744 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927746 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927749 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927752 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927756 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927758 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927761 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927765 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927767 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927770 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927773 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927775 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927778 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 17:04:39.532525 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927780 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927783 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927786 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927788 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927792 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927795 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927799 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927801 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927804 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927807 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927809 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927812 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927815 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927817 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927820 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927822 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927825 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927827 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927829 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 17:04:39.533098 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927832 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927834 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927837 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927839 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927844 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927846 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927849 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927853 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927855 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927858 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927860 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927863 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927865 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927868 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927871 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927873 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927876 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927879 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927881 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927884 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927887 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 17:04:39.533717 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927889 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927891 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927894 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927897 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927899 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927901 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927904 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927906 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927910 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927913 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927916 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927919 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927921 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927925 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927927 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927933 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 17:04:39.534297 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:38.927935 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.927940 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.928070 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.930846 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.932143 2584 server.go:1019] "Starting client certificate rotation" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.932251 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.933017 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.935327 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.937589 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.952589 2584 log.go:25] "Validated CRI v1 runtime API" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.954746 2584 log.go:25] "Validated CRI v1 image API" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.955551 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 17:04:39.534924 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.960019 2584 fs.go:135] Filesystem UUIDs: map[51c88928-a9a1-4499-842e-3cd20ecf49c2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8113146a-85e4-4063-b42f-c72a9beb4510:/dev/nvme0n1p4] Jul 23 17:04:39.535296 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.960037 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 17:04:39.535296 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.962179 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.964993 2584 manager.go:216] Machine: {Timestamp:2026-07-23 17:04:38.963711886 +0000 UTC m=+0.317573255 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204346 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2911cf912b045c435e38f74fd7a442 SystemUUID:ec2911cf-912b-045c-435e-38f74fd7a442 BootID:b8f4e4f6-0078-40f7-a874-8ba7ef8d117e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fa:bd:6e:0d:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fa:bd:6e:0d:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:b6:e6:21:b1:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.965105 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.965196 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.967892 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.967932 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-29.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.969216 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.969228 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.969240 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.969834 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.971439 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975567 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975706 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975736 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975750 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975768 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975780 2584 kubelet.go:398] "Adding apiserver pod source" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.975792 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 17:04:39.535483 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.976227 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.976247 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.978212 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.979725 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.979752 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980448 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980463 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980470 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980475 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980503 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980512 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980520 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980526 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980532 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980538 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980547 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.980552 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.981802 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.981812 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.985556 2584 server.go:1268] "Started kubelet" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.985630 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.985780 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.986035 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.986459 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-29.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:38.986534 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:38.994102 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.994556 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:38.995718 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:38.997401 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:38.998713 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-29.ec2.internal.18c4fa0b58b6b86a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-29.ec2.internal,UID:ip-10-0-136-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-29.ec2.internal,},FirstTimestamp:2026-07-23 17:04:38.985513066 +0000 UTC m=+0.339374437,LastTimestamp:2026-07-23 17:04:38.985513066 +0000 UTC m=+0.339374437,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-29.ec2.internal,}" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.002617 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.002617 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.002846 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003337 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003338 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003377 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003515 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003525 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003826 2584 factory.go:55] Registering systemd factory Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.003842 2584 factory.go:223] Registration of the systemd container factory successfully Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004089 2584 factory.go:153] Registering CRI-O factory Jul 23 17:04:39.536053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004102 2584 factory.go:223] Registration of the crio container factory successfully Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004157 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004180 2584 factory.go:103] Registering Raw factory Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004195 2584 manager.go:1196] Started watching for new ooms in manager Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.004894 2584 manager.go:349] Starting recovery of all containers Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.011438 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-29.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.011617 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.017456 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9cxrp" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.018692 2584 manager.go:354] Recovery completed Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.024931 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.025411 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9cxrp" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027112 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027144 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027157 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027739 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027746 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027747 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.027807 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.028572 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.028599 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.028609 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.029226 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-29.ec2.internal.18c4fa0b5b31b0b5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-29.ec2.internal,UID:ip-10-0-136-29.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-29.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-29.ec2.internal,},FirstTimestamp:2026-07-23 17:04:39.027126453 +0000 UTC m=+0.380987822,LastTimestamp:2026-07-23 17:04:39.027126453 +0000 UTC m=+0.380987822,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-29.ec2.internal,}" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.029978 2584 policy_none.go:49] "None policy: Start" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.029992 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.030002 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.031988 2584 policy_none.go:47] "Start" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089043 2584 manager.go:337] "Starting Device Plugin manager" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.089232 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089243 2584 server.go:87] "Starting device plugin registration server" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089511 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089528 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089619 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089699 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.089705 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.090245 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.090289 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.123115 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.124385 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.124437 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 17:04:39.801877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.124464 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.124516 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.126442 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.190348 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.191374 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.191401 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.191410 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.191432 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.201182 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.201206 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-29.ec2.internal\": node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.224695 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal"] Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.224770 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.226215 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.226241 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.226251 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.229905 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.229942 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.230612 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.230723 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.230756 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.230768 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.231227 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.231248 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.231257 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.233501 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.233525 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.234191 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientMemory" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.234213 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.234221 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeHasSufficientPID" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.250607 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.256069 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-29.ec2.internal\" not found" node="ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.260025 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-29.ec2.internal\" not found" node="ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.306378 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.306407 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.803909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.306424 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e406453e6e016770b211c4662b0c1b45-config\") pod \"kube-apiserver-proxy-ip-10-0-136-29.ec2.internal\" (UID: \"e406453e6e016770b211c4662b0c1b45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.351220 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.406982 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.407020 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.407047 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e406453e6e016770b211c4662b0c1b45-config\") pod \"kube-apiserver-proxy-ip-10-0-136-29.ec2.internal\" (UID: \"e406453e6e016770b211c4662b0c1b45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.407101 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e406453e6e016770b211c4662b0c1b45-config\") pod \"kube-apiserver-proxy-ip-10-0-136-29.ec2.internal\" (UID: \"e406453e6e016770b211c4662b0c1b45\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.407101 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.407124 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a40cbf0b87c1d58510a8fe62ed5b9dc1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal\" (UID: \"a40cbf0b87c1d58510a8fe62ed5b9dc1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.451388 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.554276 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.557661 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.564013 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.657716 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:39.805706 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.758327 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-29.ec2.internal\" not found" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.840596 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.851138 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.906196 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.922649 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.925520 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.931834 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.931992 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.932023 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.932023 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:39.932066 2584 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a152e818864384ee5a46c1a2f02c437f-0894e408cc29961f.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.136.29:41276->44.207.20.17:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:39.976393 2584 apiserver.go:52] "Watching apiserver" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.000233 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.000598 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-b7zbk","openshift-image-registry/node-ca-g7x68","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal","openshift-multus/multus-additional-cni-plugins-gz9n5","openshift-multus/multus-d7hgq","kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal","openshift-dns/node-resolver-hzlv7","openshift-multus/network-metrics-daemon-9d7qn","openshift-network-diagnostics/network-check-target-fshkv","openshift-network-operator/iptables-alerter-md77m","openshift-ovn-kubernetes/ovnkube-node-5wgfn","kube-system/global-pull-secret-syncer-g42dl","kube-system/konnectivity-agent-sl89n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg"] Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.002788 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.003341 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.005880 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.005948 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.006302 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.006408 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8q4fd\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.008754 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.008880 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9nkvn\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.009075 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.011367 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkx7\" (UniqueName: \"kubernetes.io/projected/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-kube-api-access-jdkx7\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.011753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7188a0ac-f5c3-4a35-a37a-c504a176efe0-agent-certs\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.011774 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7188a0ac-f5c3-4a35-a37a-c504a176efe0-konnectivity-ca\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.044927 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.011793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-hosts-file\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.011809 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-tmp-dir\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.013433 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.015860 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.015999 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wdkhj\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.016070 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.016177 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.016335 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.017492 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.018432 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.018736 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.018760 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.018853 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggqzl\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.019110 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.019220 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.019220 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.020712 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.020785 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.020809 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.021224 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4q96l\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.023157 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.023241 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.029125 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.031040 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 16:59:39 +0000 UTC" deadline="2027-12-23 20:50:43.40450336 +0000 UTC" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.031074 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12435h46m3.373431876s" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.031494 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.031977 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7rq6j\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.031981 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.032047 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.033782 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.033987 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.034407 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.046332 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.034410 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.034448 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c5cv9\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.036418 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.036644 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.036644 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zmzlg\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.036919 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.037213 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039633 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039688 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039728 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.039738 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039790 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039832 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r689f\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.039876 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.040865 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.040893 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 17:04:40.047534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.042273 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t4mtf" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.052380 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t4mtf" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.104626 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112576 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-conf-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-kubelet\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112629 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-sys\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112656 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-var-lib-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112682 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-script-lib\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9mn\" (UniqueName: \"kubernetes.io/projected/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kube-api-access-bz9mn\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112778 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-host\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112803 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-dbus\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112835 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-hosts-file\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112877 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-device-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112956 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-etc-kubernetes\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112958 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-hosts-file\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.112971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/497a9838-9e96-4f68-8e2a-db3b3172959e-host\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysconfig\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.272089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-kubernetes\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113062 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-run\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-765jh\" (UniqueName: \"kubernetes.io/projected/497a9838-9e96-4f68-8e2a-db3b3172959e-kube-api-access-765jh\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-os-release\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113126 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62mlq\" (UniqueName: \"kubernetes.io/projected/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-kube-api-access-62mlq\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-system-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113166 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp9s2\" (UniqueName: \"kubernetes.io/projected/7779d10e-5a02-44cf-97e8-300dd7692e76-kube-api-access-hp9s2\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-tuned\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113201 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-slash\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113217 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113251 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l62p8\" (UniqueName: \"kubernetes.io/projected/10e31c6d-240f-42c7-8952-6b10d8ece69c-kube-api-access-l62p8\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113278 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113305 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113331 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-netns\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113355 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-bin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113390 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-systemd\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.273472 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113414 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10e31c6d-240f-42c7-8952-6b10d8ece69c-iptables-alerter-script\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113436 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-netns\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113457 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-ovn\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113479 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-multus\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113546 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-modprobe-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-lib-modules\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113626 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/497a9838-9e96-4f68-8e2a-db3b3172959e-serviceca\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113662 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-log-socket\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-netd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113702 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-registration-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113719 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-hostroot\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113749 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovn-node-metrics-cert\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113772 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113787 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl6hx\" (UniqueName: \"kubernetes.io/projected/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-kube-api-access-rl6hx\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.274692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113809 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-kubelet\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113836 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113855 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6t2v\" (UniqueName: \"kubernetes.io/projected/0073ae2a-622f-4f48-b28a-ebb9afb61b95-kube-api-access-w6t2v\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113870 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113886 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-sys-fs\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-os-release\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113949 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-k8s-cni-cncf-io\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113970 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-multus-daemon-config\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.113986 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114001 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114024 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-systemd-units\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114054 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-conf\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114075 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7188a0ac-f5c3-4a35-a37a-c504a176efe0-agent-certs\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkx7\" (UniqueName: \"kubernetes.io/projected/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-kube-api-access-jdkx7\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114108 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cnibin\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.275676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114141 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114157 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-socket-dir-parent\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114182 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-etc-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114214 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7188a0ac-f5c3-4a35-a37a-c504a176efe0-konnectivity-ca\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114273 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-cni-binary-copy\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114297 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-multus-certs\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114317 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10e31c6d-240f-42c7-8952-6b10d8ece69c-host-slash\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114338 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-bin\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114386 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-env-overrides\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114405 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-tmp-dir\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114420 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-socket-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114437 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-cnibin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114469 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114489 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-var-lib-kubelet\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114522 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-tmp\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114567 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-node-log\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.276601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114588 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-config\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csqt7\" (UniqueName: \"kubernetes.io/projected/b18db2af-1afb-4965-b9d8-524a887263a6-kube-api-access-csqt7\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-systemd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114687 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-kubelet-config\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114714 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-tmp-dir\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.114817 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7188a0ac-f5c3-4a35-a37a-c504a176efe0-konnectivity-ca\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.117866 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7188a0ac-f5c3-4a35-a37a-c504a176efe0-agent-certs\") pod \"konnectivity-agent-sl89n\" (UID: \"7188a0ac-f5c3-4a35-a37a-c504a176efe0\") " pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.129252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkx7\" (UniqueName: \"kubernetes.io/projected/cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf-kube-api-access-jdkx7\") pod \"node-resolver-hzlv7\" (UID: \"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf\") " pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.214969 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215002 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cnibin\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-socket-dir-parent\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215055 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-etc-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215072 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-cni-binary-copy\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215105 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cnibin\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215145 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-socket-dir-parent\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.278619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-etc-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215172 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-multus-certs\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215186 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215199 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10e31c6d-240f-42c7-8952-6b10d8ece69c-host-slash\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215219 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-bin\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215227 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215241 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-env-overrides\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215249 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10e31c6d-240f-42c7-8952-6b10d8ece69c-host-slash\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215218 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-multus-certs\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215270 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-bin\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-socket-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215308 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-cnibin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-var-lib-kubelet\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215358 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-tmp\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215400 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-socket-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215420 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-node-log\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-config\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.279545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215452 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-var-lib-kubelet\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215406 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-cnibin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215471 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215495 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csqt7\" (UniqueName: \"kubernetes.io/projected/b18db2af-1afb-4965-b9d8-524a887263a6-kube-api-access-csqt7\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215516 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-systemd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215551 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-systemd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-kubelet-config\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-conf-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215616 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-kubelet\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215640 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-sys\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215646 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-cni-binary-copy\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-var-lib-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215695 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-node-log\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-script-lib\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215724 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-system-cni-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215742 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9mn\" (UniqueName: \"kubernetes.io/projected/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kube-api-access-bz9mn\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215766 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-host\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.280262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215788 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-dbus\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-device-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215833 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-sys\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215852 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-etc-kubernetes\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215884 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-kubelet-config\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/497a9838-9e96-4f68-8e2a-db3b3172959e-host\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215925 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-etc-kubernetes\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysconfig\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215963 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-kubernetes\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215965 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-var-lib-openvswitch\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.215789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-env-overrides\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-config\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216145 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-kubernetes\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216175 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-run\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216200 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-765jh\" (UniqueName: \"kubernetes.io/projected/497a9838-9e96-4f68-8e2a-db3b3172959e-kube-api-access-765jh\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216229 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-os-release\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62mlq\" (UniqueName: \"kubernetes.io/projected/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-kube-api-access-62mlq\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.281020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-system-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216306 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp9s2\" (UniqueName: \"kubernetes.io/projected/7779d10e-5a02-44cf-97e8-300dd7692e76-kube-api-access-hp9s2\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-tuned\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216462 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-slash\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216488 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216512 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l62p8\" (UniqueName: \"kubernetes.io/projected/10e31c6d-240f-42c7-8952-6b10d8ece69c-kube-api-access-l62p8\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216562 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-netns\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216608 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-bin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-systemd\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216653 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10e31c6d-240f-42c7-8952-6b10d8ece69c-iptables-alerter-script\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216659 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovnkube-script-lib\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216677 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-netns\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216700 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-ovn\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216726 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216733 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/403649e1-c982-4cb0-9868-4b60e570db92-dbus\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.281679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216751 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216759 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-kubelet\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216778 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-multus\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216784 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-multus-conf-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-modprobe-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216814 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysconfig\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216830 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-lib-modules\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216936 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/497a9838-9e96-4f68-8e2a-db3b3172959e-serviceca\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216958 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-log-socket\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-netd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217006 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-registration-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217027 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-hostroot\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217051 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovn-node-metrics-cert\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217066 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-host\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217074 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217098 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-device-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-run\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl6hx\" (UniqueName: \"kubernetes.io/projected/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-kube-api-access-rl6hx\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.282260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217183 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-kubelet\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217234 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6t2v\" (UniqueName: \"kubernetes.io/projected/0073ae2a-622f-4f48-b28a-ebb9afb61b95-kube-api-access-w6t2v\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217259 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217282 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-sys-fs\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217305 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-os-release\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-k8s-cni-cncf-io\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-multus-daemon-config\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217404 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217427 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-systemd-units\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217450 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217467 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-conf\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217498 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-system-cni-dir\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217578 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-sys-fs\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.217593 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217644 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-os-release\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283098 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.217667 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:40.717632714 +0000 UTC m=+2.071494125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217687 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-os-release\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.216703 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-slash\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217765 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-cni-netd\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217812 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217860 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-netns\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217902 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-bin\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217955 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217961 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-kubelet\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217955 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-systemd\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.217997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/497a9838-9e96-4f68-8e2a-db3b3172959e-host\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218019 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-log-socket\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-run-netns\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218184 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-etc-selinux\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218338 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-run-ovn\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218394 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-modprobe-d\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218422 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10e31c6d-240f-42c7-8952-6b10d8ece69c-iptables-alerter-script\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.283714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218435 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-systemd-units\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218477 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd334c83-2b5e-4db2-b7a4-d4fd56704845-registration-dir\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218514 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-hostroot\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218562 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-var-lib-cni-multus\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218620 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-lib-modules\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218661 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b18db2af-1afb-4965-b9d8-524a887263a6-host-run-k8s-cni-cncf-io\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218669 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0073ae2a-622f-4f48-b28a-ebb9afb61b95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218814 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b18db2af-1afb-4965-b9d8-524a887263a6-multus-daemon-config\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.218865 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.218870 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-sysctl-conf\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.218924 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:40.718906691 +0000 UTC m=+2.072768068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.219014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/497a9838-9e96-4f68-8e2a-db3b3172959e-serviceca\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.219274 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.220189 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-tmp\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.220217 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-etc-tuned\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.220998 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0073ae2a-622f-4f48-b28a-ebb9afb61b95-ovn-node-metrics-cert\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.462192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.251123 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csqt7\" (UniqueName: \"kubernetes.io/projected/b18db2af-1afb-4965-b9d8-524a887263a6-kube-api-access-csqt7\") pod \"multus-d7hgq\" (UID: \"b18db2af-1afb-4965-b9d8-524a887263a6\") " pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.254044 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9mn\" (UniqueName: \"kubernetes.io/projected/cd334c83-2b5e-4db2-b7a4-d4fd56704845-kube-api-access-bz9mn\") pod \"aws-ebs-csi-driver-node-xgwjg\" (UID: \"cd334c83-2b5e-4db2-b7a4-d4fd56704845\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.255703 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l62p8\" (UniqueName: \"kubernetes.io/projected/10e31c6d-240f-42c7-8952-6b10d8ece69c-kube-api-access-l62p8\") pod \"iptables-alerter-md77m\" (UID: \"10e31c6d-240f-42c7-8952-6b10d8ece69c\") " pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.263519 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62mlq\" (UniqueName: \"kubernetes.io/projected/db97698d-6bc7-42ad-b7e7-b6e435f81d0a-kube-api-access-62mlq\") pod \"multus-additional-cni-plugins-gz9n5\" (UID: \"db97698d-6bc7-42ad-b7e7-b6e435f81d0a\") " pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.265799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl6hx\" (UniqueName: \"kubernetes.io/projected/bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166-kube-api-access-rl6hx\") pod \"tuned-b7zbk\" (UID: \"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166\") " pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.266589 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.266608 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.266620 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.266682 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:04:40.766661912 +0000 UTC m=+2.120523271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.267876 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-765jh\" (UniqueName: \"kubernetes.io/projected/497a9838-9e96-4f68-8e2a-db3b3172959e-kube-api-access-765jh\") pod \"node-ca-g7x68\" (UID: \"497a9838-9e96-4f68-8e2a-db3b3172959e\") " pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.269298 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6t2v\" (UniqueName: \"kubernetes.io/projected/0073ae2a-622f-4f48-b28a-ebb9afb61b95-kube-api-access-w6t2v\") pod \"ovnkube-node-5wgfn\" (UID: \"0073ae2a-622f-4f48-b28a-ebb9afb61b95\") " pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.272852 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp9s2\" (UniqueName: \"kubernetes.io/projected/7779d10e-5a02-44cf-97e8-300dd7692e76-kube-api-access-hp9s2\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.291707 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40cbf0b87c1d58510a8fe62ed5b9dc1.slice/crio-2632d4a8b275d0c95d0a424b5020f1bf5a503fae4f3da92eac2f00932592fae5 WatchSource:0}: Error finding container 2632d4a8b275d0c95d0a424b5020f1bf5a503fae4f3da92eac2f00932592fae5: Status 404 returned error can't find the container with id 2632d4a8b275d0c95d0a424b5020f1bf5a503fae4f3da92eac2f00932592fae5 Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.295416 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.334994 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hzlv7" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.346336 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.348253 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5cb2c4_fa6e_4c4a_a487_a0b01f5fc7cf.slice/crio-0d24f8daa87a3eae8f6fb1cee756baa5abb2eaf6f0a4017d0ca73a3df1b34b08 WatchSource:0}: Error finding container 0d24f8daa87a3eae8f6fb1cee756baa5abb2eaf6f0a4017d0ca73a3df1b34b08: Status 404 returned error can't find the container with id 0d24f8daa87a3eae8f6fb1cee756baa5abb2eaf6f0a4017d0ca73a3df1b34b08 Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.363188 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7188a0ac_f5c3_4a35_a37a_c504a176efe0.slice/crio-79446384edce4a913193b5b12104267c39d9ff86358f024ea89221b79a9c4be2 WatchSource:0}: Error finding container 79446384edce4a913193b5b12104267c39d9ff86358f024ea89221b79a9c4be2: Status 404 returned error can't find the container with id 79446384edce4a913193b5b12104267c39d9ff86358f024ea89221b79a9c4be2 Jul 23 17:04:40.464601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.369881 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.381238 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode406453e6e016770b211c4662b0c1b45.slice/crio-4ff6232836736cfd38043ec67cb6494b0d84e7ddf304f1a19516691609aeee02 WatchSource:0}: Error finding container 4ff6232836736cfd38043ec67cb6494b0d84e7ddf304f1a19516691609aeee02: Status 404 returned error can't find the container with id 4ff6232836736cfd38043ec67cb6494b0d84e7ddf304f1a19516691609aeee02 Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.382653 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd334c83_2b5e_4db2_b7a4_d4fd56704845.slice/crio-f26e0efc79dfc397061c402a4a21a789f017265e3e32a73b42be6ccfd553d52d WatchSource:0}: Error finding container f26e0efc79dfc397061c402a4a21a789f017265e3e32a73b42be6ccfd553d52d: Status 404 returned error can't find the container with id f26e0efc79dfc397061c402a4a21a789f017265e3e32a73b42be6ccfd553d52d Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.387157 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.402059 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb97698d_6bc7_42ad_b7e7_b6e435f81d0a.slice/crio-1e7b8652a0dfd89e498b9094018e261947e58cd7404e21f3cd5d8bf7d2c81467 WatchSource:0}: Error finding container 1e7b8652a0dfd89e498b9094018e261947e58cd7404e21f3cd5d8bf7d2c81467: Status 404 returned error can't find the container with id 1e7b8652a0dfd89e498b9094018e261947e58cd7404e21f3cd5d8bf7d2c81467 Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.408099 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7hgq" Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.417219 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18db2af_1afb_4965_b9d8_524a887263a6.slice/crio-927926ecdc435c2cdfb7e3e0e9a30b2e8d5b6b86d6a099928960b87c1e4384e8 WatchSource:0}: Error finding container 927926ecdc435c2cdfb7e3e0e9a30b2e8d5b6b86d6a099928960b87c1e4384e8: Status 404 returned error can't find the container with id 927926ecdc435c2cdfb7e3e0e9a30b2e8d5b6b86d6a099928960b87c1e4384e8 Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.427538 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.436438 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2f7a01_1acb_46cc_bb2a_3c6b34e0d166.slice/crio-07cf5eb178cb874c7c5768a2bad4a5721fb51a89b42509fdab3deee39c942222 WatchSource:0}: Error finding container 07cf5eb178cb874c7c5768a2bad4a5721fb51a89b42509fdab3deee39c942222: Status 404 returned error can't find the container with id 07cf5eb178cb874c7c5768a2bad4a5721fb51a89b42509fdab3deee39c942222 Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.444551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g7x68" Jul 23 17:04:40.465941 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.454554 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod497a9838_9e96_4f68_8e2a_db3b3172959e.slice/crio-0c35f4e865c5565eac6955e0c60218b078ab4c863ebe44cf9a99499c63469aef WatchSource:0}: Error finding container 0c35f4e865c5565eac6955e0c60218b078ab4c863ebe44cf9a99499c63469aef: Status 404 returned error can't find the container with id 0c35f4e865c5565eac6955e0c60218b078ab4c863ebe44cf9a99499c63469aef Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.469649 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-md77m" Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.477420 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.480227 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e31c6d_240f_42c7_8952_6b10d8ece69c.slice/crio-464fa7d712e063fab54baa6129b626d1ae8bde3a00997906d8617ccf3a51e92a WatchSource:0}: Error finding container 464fa7d712e063fab54baa6129b626d1ae8bde3a00997906d8617ccf3a51e92a: Status 404 returned error can't find the container with id 464fa7d712e063fab54baa6129b626d1ae8bde3a00997906d8617ccf3a51e92a Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:04:40.488603 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0073ae2a_622f_4f48_b28a_ebb9afb61b95.slice/crio-ce4312e988139154450b111aedd00c4f5ae0d7e89118180ef3521b75805e32b8 WatchSource:0}: Error finding container ce4312e988139154450b111aedd00c4f5ae0d7e89118180ef3521b75805e32b8: Status 404 returned error can't find the container with id ce4312e988139154450b111aedd00c4f5ae0d7e89118180ef3521b75805e32b8 Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.488931 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.490484 2584 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-5wgfn_openshift-ovn-kubernetes(0073ae2a-622f-4f48-b28a-ebb9afb61b95): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.491888 2584 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-5wgfn_openshift-ovn-kubernetes(0073ae2a-622f-4f48-b28a-ebb9afb61b95): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 23 17:04:40.713298 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.500139 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:534ab2ea2f6c22eaf4d2267274394590a29319c3d5e0da72bef87bc86f428dec\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" podUID="0073ae2a-622f-4f48-b28a-ebb9afb61b95" Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.720937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.721015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.721097 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.721102 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.721156 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:41.721142223 +0000 UTC m=+3.075003582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.721177 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:41.721163866 +0000 UTC m=+3.075025228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:40.821518 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.821675 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.821692 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.821704 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:40.918583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:40.821757 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:04:41.821742682 +0000 UTC m=+3.175604042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:41.106681 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.056986 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 16:59:40 +0000 UTC" deadline="2028-05-07 05:07:37.77203264 +0000 UTC" Jul 23 17:04:41.106681 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.057029 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15684h2m56.715009053s" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.229369 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.229520 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.229686 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.229797 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.234269 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" event={"ID":"cd334c83-2b5e-4db2-b7a4-d4fd56704845","Type":"ContainerStarted","Data":"f26e0efc79dfc397061c402a4a21a789f017265e3e32a73b42be6ccfd553d52d"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.241714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" event={"ID":"e406453e6e016770b211c4662b0c1b45","Type":"ContainerStarted","Data":"4ff6232836736cfd38043ec67cb6494b0d84e7ddf304f1a19516691609aeee02"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.245907 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sl89n" event={"ID":"7188a0ac-f5c3-4a35-a37a-c504a176efe0","Type":"ContainerStarted","Data":"79446384edce4a913193b5b12104267c39d9ff86358f024ea89221b79a9c4be2"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.263354 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"ce4312e988139154450b111aedd00c4f5ae0d7e89118180ef3521b75805e32b8"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.267196 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-md77m" event={"ID":"10e31c6d-240f-42c7-8952-6b10d8ece69c","Type":"ContainerStarted","Data":"464fa7d712e063fab54baa6129b626d1ae8bde3a00997906d8617ccf3a51e92a"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.283701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hzlv7" event={"ID":"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf","Type":"ContainerStarted","Data":"0d24f8daa87a3eae8f6fb1cee756baa5abb2eaf6f0a4017d0ca73a3df1b34b08"} Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.288877 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:534ab2ea2f6c22eaf4d2267274394590a29319c3d5e0da72bef87bc86f428dec\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:534ab2ea2f6c22eaf4d2267274394590a29319c3d5e0da72bef87bc86f428dec\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:241e47351d3f7eef8ef00c2c9acba53c54ac799f4e751de5b4d4e13016eaa997\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" podUID="0073ae2a-622f-4f48-b28a-ebb9afb61b95" Jul 23 17:04:41.290904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.290165 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" event={"ID":"a40cbf0b87c1d58510a8fe62ed5b9dc1","Type":"ContainerStarted","Data":"2632d4a8b275d0c95d0a424b5020f1bf5a503fae4f3da92eac2f00932592fae5"} Jul 23 17:04:41.295262 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.292747 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7x68" event={"ID":"497a9838-9e96-4f68-8e2a-db3b3172959e","Type":"ContainerStarted","Data":"0c35f4e865c5565eac6955e0c60218b078ab4c863ebe44cf9a99499c63469aef"} Jul 23 17:04:41.358911 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.297547 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" event={"ID":"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166","Type":"ContainerStarted","Data":"07cf5eb178cb874c7c5768a2bad4a5721fb51a89b42509fdab3deee39c942222"} Jul 23 17:04:41.358911 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.300893 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7hgq" event={"ID":"b18db2af-1afb-4965-b9d8-524a887263a6","Type":"ContainerStarted","Data":"927926ecdc435c2cdfb7e3e0e9a30b2e8d5b6b86d6a099928960b87c1e4384e8"} Jul 23 17:04:41.358911 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.303040 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerStarted","Data":"1e7b8652a0dfd89e498b9094018e261947e58cd7404e21f3cd5d8bf7d2c81467"} Jul 23 17:04:41.442679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.442625 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:41.607840 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.607753 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:41.729024 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.728981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:41.729235 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.729085 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:41.729297 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.729232 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:41.729356 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.729303 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:43.729283649 +0000 UTC m=+5.083145013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:41.730080 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.729509 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:41.730080 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.729581 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:43.729562038 +0000 UTC m=+5.083423415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:41.830180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:41.830112 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:41.830490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.830296 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:41.830490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.830317 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:41.830490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.830331 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:41.830490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:41.830396 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:04:43.830376807 +0000 UTC m=+5.184238182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:42.057716 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:42.057545 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 16:59:40 +0000 UTC" deadline="2028-03-28 12:28:04.848135313 +0000 UTC" Jul 23 17:04:42.057716 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:42.057615 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14731h23m22.790525547s" Jul 23 17:04:42.124979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:42.124947 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:42.125583 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:42.125149 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:42.241917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:42.241675 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 17:04:43.131446 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:43.130679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:43.131446 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.130825 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:43.131446 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:43.131240 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:43.131446 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.131377 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:43.747968 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:43.747884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:43.747968 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:43.747952 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:43.748190 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.748141 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:43.748242 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.748217 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:47.748196618 +0000 UTC m=+9.102057989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:43.748242 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.748216 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:43.748348 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.748273 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:47.748257729 +0000 UTC m=+9.102119090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:43.848961 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:43.848909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:43.849165 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.849122 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:43.849165 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.849144 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:43.849165 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.849157 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:43.849347 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:43.849215 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:04:47.849196807 +0000 UTC m=+9.203058209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:44.125314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:44.125217 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:44.125490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:44.125378 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:45.125293 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:45.125257 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:45.125837 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:45.125420 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:45.125837 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:45.125774 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:45.125944 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:45.125868 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:46.125522 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:46.125485 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:46.125987 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:46.125615 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:47.125152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:47.125107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:47.125313 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.125244 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:47.125313 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:47.125299 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:47.125459 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.125424 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:47.779647 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:47.779395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:47.779647 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:47.779483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:47.779647 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.779576 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:47.779647 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.779589 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:47.780209 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.779661 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:55.779641626 +0000 UTC m=+17.133502999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:47.780209 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.779691 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:04:55.779681935 +0000 UTC m=+17.133543293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:47.880052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:47.880009 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:47.880222 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.880173 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:47.880222 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.880193 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:47.880222 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.880206 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:47.880385 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:47.880271 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:04:55.88025287 +0000 UTC m=+17.234114251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:48.125324 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:48.125214 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:48.125513 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:48.125369 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:49.126220 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:49.126086 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:49.126220 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:49.126139 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:49.126220 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:49.126212 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:49.126780 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:49.126251 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:50.125567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:50.125488 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:50.125784 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:50.125659 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:51.124950 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:51.124909 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:51.125394 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:51.124918 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:51.125394 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:51.125028 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:51.125394 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:51.125110 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:52.125257 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:52.125221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:52.125705 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:52.125370 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:53.125300 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:53.125221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:53.125750 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:53.125347 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:53.125750 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:53.125389 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:53.125750 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:53.125509 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:54.125533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.125457 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:54.125992 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:54.125618 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:54.338267 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.338224 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" event={"ID":"bb2f7a01-1acb-46cc-bb2a-3c6b34e0d166","Type":"ContainerStarted","Data":"0e15a9ce75befbabeeb3fe5bcff55631a0882d20c8d9750799907c6c2cf4e350"} Jul 23 17:04:54.343429 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.343155 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7hgq" event={"ID":"b18db2af-1afb-4965-b9d8-524a887263a6","Type":"ContainerStarted","Data":"f54bb12de839a44c8c12c99c67d064e8cace6dd2f0eb6a4c7fc2f475a265a365"} Jul 23 17:04:54.347545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.347234 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" event={"ID":"e406453e6e016770b211c4662b0c1b45","Type":"ContainerStarted","Data":"c2eff003c45fe925c4ce2915bbe7ac6341dae67bf9a4be8a4c32942aee2229d7"} Jul 23 17:04:54.357585 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.357499 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b7zbk" podStartSLOduration=2.392654619 podStartE2EDuration="15.357475462s" totalImagesPullingTime="12.964820843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:54.356281724 +0000 UTC m=+15.710143119" watchObservedRunningTime="2026-07-23 17:04:54.357475462 +0000 UTC m=+15.711336871" Jul 23 17:04:54.372207 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:54.371947 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-29.ec2.internal" podStartSLOduration=14.371924065 podStartE2EDuration="14.371924065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:54.371094048 +0000 UTC m=+15.724955426" watchObservedRunningTime="2026-07-23 17:04:54.371924065 +0000 UTC m=+15.725785447" Jul 23 17:04:55.125285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.125030 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:55.125536 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.125047 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:55.125536 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.125403 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:55.125536 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.125497 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:55.351806 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.351583 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g7x68" event={"ID":"497a9838-9e96-4f68-8e2a-db3b3172959e","Type":"ContainerStarted","Data":"8d5c3d61558dfdb6cf7502ac004a8be2a5eb24b5ae4a2a3296d7a33cedff3371"} Jul 23 17:04:55.353876 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.353838 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="b3c6846ce1ce3bf7a5bc2a80f11d9e5af97b19cb218eaf8a81722dfa4c560d2a" exitCode=0 Jul 23 17:04:55.354014 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.353944 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"b3c6846ce1ce3bf7a5bc2a80f11d9e5af97b19cb218eaf8a81722dfa4c560d2a"} Jul 23 17:04:55.356085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.356012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" event={"ID":"cd334c83-2b5e-4db2-b7a4-d4fd56704845","Type":"ContainerStarted","Data":"4a3409369b2615c31f17baa43a43519011e704e08a2b128b4f979ff3d3da4a73"} Jul 23 17:04:55.357811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.357701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sl89n" event={"ID":"7188a0ac-f5c3-4a35-a37a-c504a176efe0","Type":"ContainerStarted","Data":"0db64e5fcb5e3a309d3f496131edaf5ea6abf2752f6276af636cdc3aab08595d"} Jul 23 17:04:55.359684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.359655 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-md77m" event={"ID":"10e31c6d-240f-42c7-8952-6b10d8ece69c","Type":"ContainerStarted","Data":"c607c498ef7732853b7f4b64dee06cef43f4d75ee7c3c7840b48f33bfe1472b2"} Jul 23 17:04:55.361799 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.361437 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hzlv7" event={"ID":"cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf","Type":"ContainerStarted","Data":"d3a239154ead7fa54e53ed02065c26db4ccd80837d404a354a4c4ddbc20b8ef5"} Jul 23 17:04:55.363181 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.363152 2584 generic.go:356] "Generic (PLEG): container finished" podID="a40cbf0b87c1d58510a8fe62ed5b9dc1" containerID="a851640610f3c7efdc24915742a1803e4150b42363751f610d769d286661c0a9" exitCode=0 Jul 23 17:04:55.363314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.363296 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" event={"ID":"a40cbf0b87c1d58510a8fe62ed5b9dc1","Type":"ContainerDied","Data":"a851640610f3c7efdc24915742a1803e4150b42363751f610d769d286661c0a9"} Jul 23 17:04:55.369014 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.368947 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d7hgq" podStartSLOduration=3.060714961 podStartE2EDuration="16.368929902s" totalImagesPullingTime="13.308214941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:54.399405525 +0000 UTC m=+15.753266908" watchObservedRunningTime="2026-07-23 17:04:55.368929902 +0000 UTC m=+16.722791260" Jul 23 17:04:55.371895 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.370300 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g7x68" podStartSLOduration=3.426336155 podStartE2EDuration="16.370281287s" totalImagesPullingTime="12.943945132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:55.368301692 +0000 UTC m=+16.722163072" watchObservedRunningTime="2026-07-23 17:04:55.370281287 +0000 UTC m=+16.724142667" Jul 23 17:04:55.428881 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.428763 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-md77m" podStartSLOduration=3.50474979 podStartE2EDuration="16.428737565s" totalImagesPullingTime="12.923987775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:55.428724658 +0000 UTC m=+16.782586039" watchObservedRunningTime="2026-07-23 17:04:55.428737565 +0000 UTC m=+16.782598949" Jul 23 17:04:55.466448 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.466389 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sl89n" podStartSLOduration=3.497220613 podStartE2EDuration="16.466349122s" totalImagesPullingTime="12.969128509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:55.46629915 +0000 UTC m=+16.820160531" watchObservedRunningTime="2026-07-23 17:04:55.466349122 +0000 UTC m=+16.820210503" Jul 23 17:04:55.467001 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.466960 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hzlv7" podStartSLOduration=3.439276629 podStartE2EDuration="16.466946863s" totalImagesPullingTime="13.027670234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:55.447855513 +0000 UTC m=+16.801716895" watchObservedRunningTime="2026-07-23 17:04:55.466946863 +0000 UTC m=+16.820808243" Jul 23 17:04:55.841721 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.841634 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:55.841987 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.841773 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:55.841987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.841804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:55.841987 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.841849 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:11.841829385 +0000 UTC m=+33.195690768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:04:55.841987 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.841912 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:55.841987 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.841972 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:11.841947207 +0000 UTC m=+33.195808565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:04:55.942435 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:55.942353 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:55.942631 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.942520 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:04:55.942631 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.942549 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:04:55.942631 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.942563 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:55.942631 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:55.942622 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:05:11.942606696 +0000 UTC m=+33.296468074 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:04:56.114764 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.114674 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:56.125071 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.125028 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:56.125255 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:56.125188 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:56.260282 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.260239 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 17:04:56.368186 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.368090 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" event={"ID":"a40cbf0b87c1d58510a8fe62ed5b9dc1","Type":"ContainerStarted","Data":"499d1f1ca57f77e27d790f607dfec5f096ac9d9f10f4aec6a32143363d3c6d4c"} Jul 23 17:04:56.370418 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.370346 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" event={"ID":"cd334c83-2b5e-4db2-b7a4-d4fd56704845","Type":"ContainerStarted","Data":"e590b592fd9d6b04a9e86921e84e9556c922624e69397baf50e58cb69f8393c1"} Jul 23 17:04:56.385654 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:56.385588 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-29.ec2.internal" podStartSLOduration=16.385562979 podStartE2EDuration="16.385562979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:56.384881454 +0000 UTC m=+17.738742835" watchObservedRunningTime="2026-07-23 17:04:56.385562979 +0000 UTC m=+17.739424361" Jul 23 17:04:57.100131 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:57.099895 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T17:04:56.260263824Z","UUID":"0b975123-4509-4772-a857-e25b44b78093","Handler":null,"Name":"","Endpoint":""} Jul 23 17:04:57.101944 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:57.101915 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 17:04:57.102099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:57.101955 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 17:04:57.125196 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:57.125155 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:57.125398 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:57.125165 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:57.125398 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:57.125299 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:57.125398 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:57.125354 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:58.125531 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:58.125490 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:04:58.126185 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:58.125652 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:04:58.377752 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:58.377653 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" event={"ID":"cd334c83-2b5e-4db2-b7a4-d4fd56704845","Type":"ContainerStarted","Data":"b4bb1c3b20c91b44b78df2d63dfd8c5883792f798a34db8ecfd035ca953dd888"} Jul 23 17:04:58.398134 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:58.398067 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xgwjg" podStartSLOduration=3.698895782 podStartE2EDuration="19.398044147s" totalImagesPullingTime="15.699148365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:04:58.397561344 +0000 UTC m=+19.751422747" watchObservedRunningTime="2026-07-23 17:04:58.398044147 +0000 UTC m=+19.751905529" Jul 23 17:04:59.125956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:59.125696 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:04:59.126427 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:59.125755 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:04:59.126427 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:59.126046 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:04:59.126427 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:04:59.126145 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:04:59.736303 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:59.736264 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:04:59.737482 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:04:59.737450 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:05:00.125489 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:00.125455 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:00.125663 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:00.125608 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:00.385176 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:00.385084 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="0bbece0e768236eb3bfdd30f30c1bf26d1c8dcb2e83735039b451932f12f17f3" exitCode=0 Jul 23 17:05:00.385176 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:00.385166 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"0bbece0e768236eb3bfdd30f30c1bf26d1c8dcb2e83735039b451932f12f17f3"} Jul 23 17:05:00.385945 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:00.385820 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sl89n" Jul 23 17:05:01.125018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:01.124978 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:01.125212 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:01.125108 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:01.125212 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:01.125162 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:01.125323 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:01.125274 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:02.125568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:02.125534 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:02.126043 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:02.125687 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:03.125636 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:03.125599 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:03.126205 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:03.125745 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:03.126205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:03.125777 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:03.126205 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:03.125883 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:04.125246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:04.125213 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:04.125440 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:04.125370 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:05.124900 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:05.124864 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:05.125296 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:05.124973 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:05.125296 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:05.125029 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:05.125296 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:05.125121 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:06.125621 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.125423 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:06.125983 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:06.125739 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:06.399956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.399920 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"f7fe2a5fee44fae10ee140e482ed144d115922863177246d5ae1d811f47ee3ed"} Jul 23 17:05:06.400180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.400015 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"faed92a3422b9da55e3b4cfa7c9a9a74d7e9a71b1a77adf00ba82a2e0fbfdb7b"} Jul 23 17:05:06.400180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.400072 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"11fd80198c562eb71fe9854ce95249546281c0c0f192b75271929416902b391d"} Jul 23 17:05:06.400180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.400119 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"210bda3ceefc4f9b85676e36e7ed3a1e4d3d02ef3d7bd9510f63962e022f4f68"} Jul 23 17:05:06.400180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.400155 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"62f81caee3228c8d40ddd523ad4ef5d13242a68f05a224ce286a9b93821e1492"} Jul 23 17:05:06.400335 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.400186 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"a1110b713db7d638fbfb8c05294b5e48bb7ab5760c7e7f18e993acd4dd7e62cd"} Jul 23 17:05:06.401943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.401919 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="82d63ccb0435a90cbb003abbab7978179d7a47e6d9fd62f947484a18100104fb" exitCode=0 Jul 23 17:05:06.402052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:06.401950 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"82d63ccb0435a90cbb003abbab7978179d7a47e6d9fd62f947484a18100104fb"} Jul 23 17:05:07.124690 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:07.124648 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:07.124836 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:07.124659 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:07.124836 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:07.124773 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:07.124836 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:07.124824 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:08.125408 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:08.125304 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:08.125878 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:08.125459 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:08.410299 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:08.410265 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="11f8cbbf868980cb8d155d7dea7079a9e238a5ba0d34df25c8a448ce1178f813" exitCode=0 Jul 23 17:05:08.410424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:08.410343 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"11f8cbbf868980cb8d155d7dea7079a9e238a5ba0d34df25c8a448ce1178f813"} Jul 23 17:05:09.128460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:09.128425 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:09.128460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:09.128449 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:09.129218 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:09.128549 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:09.129218 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:09.128683 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:09.417587 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:09.417500 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"28ff59571a2406d08aa4083b1ce72c17cf4cb2f79fa55e70a82884bb9f34da48"} Jul 23 17:05:10.125161 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:10.125127 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:10.125340 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:10.125254 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:11.128770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.128551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:11.129425 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.128632 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:11.129425 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.128934 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:11.129425 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.129068 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:11.426598 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.426515 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" event={"ID":"0073ae2a-622f-4f48-b28a-ebb9afb61b95","Type":"ContainerStarted","Data":"c829d8cf054c8207117d91aeccd791463e0ed87ed32886d2a0dcddbcdb8e7e42"} Jul 23 17:05:11.426846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.426815 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:11.445283 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.445244 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:11.456865 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.456792 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" podStartSLOduration=7.470554631 podStartE2EDuration="32.456776672s" totalImagesPullingTime="24.986222041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-23 17:05:11.45654142 +0000 UTC m=+32.810402804" watchObservedRunningTime="2026-07-23 17:05:11.456776672 +0000 UTC m=+32.810638051" Jul 23 17:05:11.856969 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.856930 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:11.857164 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.856994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:11.857164 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.857131 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:05:11.857271 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.857193 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs podName:7779d10e-5a02-44cf-97e8-300dd7692e76 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:43.857174463 +0000 UTC m=+65.211035837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs") pod "network-metrics-daemon-9d7qn" (UID: "7779d10e-5a02-44cf-97e8-300dd7692e76") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 17:05:11.857271 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.857248 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 17:05:11.857393 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.857278 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret podName:403649e1-c982-4cb0-9868-4b60e570db92 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:43.857268919 +0000 UTC m=+65.211130290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret") pod "global-pull-secret-syncer-g42dl" (UID: "403649e1-c982-4cb0-9868-4b60e570db92") : object "kube-system"/"original-pull-secret" not registered Jul 23 17:05:11.957711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:11.957671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:11.957896 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.957810 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 17:05:11.957896 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.957831 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 17:05:11.957896 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.957843 2584 projected.go:196] Error preparing data for projected volume kube-api-access-kkxdm for pod openshift-network-diagnostics/network-check-target-fshkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:05:11.958027 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:11.957903 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm podName:64ce3378-8e7f-4b14-a25d-f77329f8e45a nodeName:}" failed. No retries permitted until 2026-07-23 17:05:43.957889315 +0000 UTC m=+65.311750672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kkxdm" (UniqueName: "kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm") pod "network-check-target-fshkv" (UID: "64ce3378-8e7f-4b14-a25d-f77329f8e45a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 17:05:12.124865 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.124760 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:12.125028 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:12.124917 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:12.429344 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.429260 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:12.429344 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.429328 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:12.445541 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.445501 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:12.791135 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.791099 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g42dl"] Jul 23 17:05:12.791314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.791243 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:12.791384 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:12.791330 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:12.795537 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.795511 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9d7qn"] Jul 23 17:05:12.795688 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.795659 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:12.795774 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:12.795754 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:12.804077 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.804049 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fshkv"] Jul 23 17:05:12.804208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:12.804158 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:12.804261 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:12.804242 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:13.433950 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:13.433917 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="aed71842303bc9f0e0573a35a08a82bf650b20b9d2ef4019426475c19ec3c712" exitCode=0 Jul 23 17:05:13.434456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:13.433998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"aed71842303bc9f0e0573a35a08a82bf650b20b9d2ef4019426475c19ec3c712"} Jul 23 17:05:14.125094 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:14.125052 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:14.125303 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:14.125052 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:14.125303 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:14.125173 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:14.125303 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:14.125252 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:14.438811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:14.438779 2584 generic.go:356] "Generic (PLEG): container finished" podID="db97698d-6bc7-42ad-b7e7-b6e435f81d0a" containerID="a62b01182337f678bb354ca929dcbaa7b9a32d6adcfff722cc17853327d77aab" exitCode=0 Jul 23 17:05:14.439264 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:14.438850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerDied","Data":"a62b01182337f678bb354ca929dcbaa7b9a32d6adcfff722cc17853327d77aab"} Jul 23 17:05:15.128371 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:15.128321 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:15.128544 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:15.128445 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:15.444650 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:15.444444 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" event={"ID":"db97698d-6bc7-42ad-b7e7-b6e435f81d0a","Type":"ContainerStarted","Data":"edf4ce76bde550a14a27e1b3fcb0eb96b875ec0650d946490775a0e83dde6dc7"} Jul 23 17:05:15.474415 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:15.474345 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gz9n5" podStartSLOduration=7.771442231 podStartE2EDuration="36.474325268s" totalImagesPullingTime="28.702883037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:15.472924022 +0000 UTC m=+36.826785398" watchObservedRunningTime="2026-07-23 17:05:15.474325268 +0000 UTC m=+36.828186646" Jul 23 17:05:16.125035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:16.124996 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:16.125216 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:16.125099 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:16.125216 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:16.125105 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:16.125354 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:16.125230 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:17.125755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:17.125674 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:17.126100 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:17.125778 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fshkv" podUID="64ce3378-8e7f-4b14-a25d-f77329f8e45a" Jul 23 17:05:18.124716 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:18.124669 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:18.124917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:18.124669 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:18.124917 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:18.124839 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9d7qn" podUID="7779d10e-5a02-44cf-97e8-300dd7692e76" Jul 23 17:05:18.124917 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:18.124869 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g42dl" podUID="403649e1-c982-4cb0-9868-4b60e570db92" Jul 23 17:05:19.008469 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.008442 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-29.ec2.internal" event="NodeReady" Jul 23 17:05:19.008848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.008559 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 17:05:19.047407 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.047351 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv"] Jul 23 17:05:19.062928 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.062897 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9f9cf5b5b-kqk57"] Jul 23 17:05:19.063112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.063022 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.065579 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.065553 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 17:05:19.066267 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.066238 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.067430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.067413 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.067637 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.067620 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kxsgq\"" Jul 23 17:05:19.067716 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.067685 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 17:05:19.088870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.087738 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:05:19.088870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.088445 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.095055 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.095026 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 17:05:19.096481 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.096459 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 17:05:19.096988 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.096635 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.096988 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.096843 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 17:05:19.096988 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.096926 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.098092 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.097699 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 17:05:19.098717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.098399 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7c5t4\"" Jul 23 17:05:19.103798 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.103759 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz"] Jul 23 17:05:19.103953 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.103893 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.107759 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.107737 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 17:05:19.109726 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.109696 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 17:05:19.109956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.109937 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 17:05:19.110210 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.110173 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wgjfb\"" Jul 23 17:05:19.115047 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.115018 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 17:05:19.120115 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.120085 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd"] Jul 23 17:05:19.120267 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.120183 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.123422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.123394 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 17:05:19.123549 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.123476 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 17:05:19.123643 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.123626 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.123773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.123754 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4wgsm\"" Jul 23 17:05:19.123861 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.123846 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.145639 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.145611 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" Jul 23 17:05:19.145817 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.145615 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:19.148478 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148455 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.148623 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148595 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.148709 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148677 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.148870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148846 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-z4xld\"" Jul 23 17:05:19.148973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148899 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fr6xr\"" Jul 23 17:05:19.148973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.148943 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.149444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.149428 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-82hnk"] Jul 23 17:05:19.164988 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.164959 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kfj29"] Jul 23 17:05:19.165146 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.165070 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.168238 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.168219 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 17:05:19.168543 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.168528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bg4w4\"" Jul 23 17:05:19.168906 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.168889 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 17:05:19.168987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.168933 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.169194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.169175 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.175981 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.175895 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 17:05:19.181323 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.181289 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2"] Jul 23 17:05:19.181491 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.181431 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.184242 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.184219 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 17:05:19.184417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.184220 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.184417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.184331 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 17:05:19.184572 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.184559 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2p74p\"" Jul 23 17:05:19.184993 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.184975 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.189782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.189760 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 17:05:19.203722 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.203688 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl"] Jul 23 17:05:19.203849 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.203815 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.206388 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.206345 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.206533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.206419 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zbwwq\"" Jul 23 17:05:19.206533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.206497 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 17:05:19.206533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.206499 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.206684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.206608 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 17:05:19.209919 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.209891 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.209923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.209951 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hbcd\" (UniqueName: \"kubernetes.io/projected/8f904527-8175-4f4c-8a6d-cd33aad0fc65-kube-api-access-8hbcd\") pod \"volume-data-source-validator-78cf4c6f6-gc7kd\" (UID: \"8f904527-8175-4f4c-8a6d-cd33aad0fc65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.209971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.209988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210009 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210030 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.210051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210045 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v66fv\" (UniqueName: \"kubernetes.io/projected/3021ee46-d3b1-4152-bec2-24e89cabe038-kube-api-access-v66fv\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210150 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210174 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210246 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-stats-auth\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210267 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mshv\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210292 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3582e243-6f99-400c-b7e4-e4847ba68e9b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwb6\" (UniqueName: \"kubernetes.io/projected/3582e243-6f99-400c-b7e4-e4847ba68e9b-kube-api-access-gcwb6\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210343 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f82b19-2fbb-489f-a569-75d296e7f85c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210387 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-default-certificate\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.210423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210412 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qj8q\" (UniqueName: \"kubernetes.io/projected/e9f82b19-2fbb-489f-a569-75d296e7f85c-kube-api-access-5qj8q\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.211030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.210439 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f82b19-2fbb-489f-a569-75d296e7f85c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.225534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.225507 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr"] Jul 23 17:05:19.225667 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.225619 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" Jul 23 17:05:19.228078 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.228059 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xpg7t\"" Jul 23 17:05:19.251639 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.251587 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv"] Jul 23 17:05:19.251639 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.251632 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2"] Jul 23 17:05:19.251639 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.251649 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p8jvx"] Jul 23 17:05:19.251874 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.251751 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.254735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.254704 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 17:05:19.255194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.255179 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6mqhb\"" Jul 23 17:05:19.255343 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.255327 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.255422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.255388 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.267557 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.267480 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bzknq"] Jul 23 17:05:19.267732 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.267615 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.270405 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.270381 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 17:05:19.270542 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.270464 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5n9dp\"" Jul 23 17:05:19.270542 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.270464 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 17:05:19.281562 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281538 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-82hnk"] Jul 23 17:05:19.281693 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281574 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kfj29"] Jul 23 17:05:19.281693 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281592 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl"] Jul 23 17:05:19.281693 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281613 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:05:19.281693 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281632 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9f9cf5b5b-kqk57"] Jul 23 17:05:19.281693 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281647 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.281926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281651 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr"] Jul 23 17:05:19.281926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281826 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd"] Jul 23 17:05:19.281926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281854 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzknq"] Jul 23 17:05:19.281926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281880 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz"] Jul 23 17:05:19.281926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.281898 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p8jvx"] Jul 23 17:05:19.284217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.284197 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 17:05:19.284394 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.284276 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 17:05:19.284394 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.284290 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j8wcn\"" Jul 23 17:05:19.284394 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.284300 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 17:05:19.310818 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310780 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j6vb\" (UniqueName: \"kubernetes.io/projected/1ef33513-8b26-482f-84eb-20918d4d1b74-kube-api-access-2j6vb\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310834 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mshv\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310857 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-config\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310875 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qj8q\" (UniqueName: \"kubernetes.io/projected/e9f82b19-2fbb-489f-a569-75d296e7f85c-kube-api-access-5qj8q\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f82b19-2fbb-489f-a569-75d296e7f85c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310925 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310951 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-serving-cert\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.310987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.310975 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6tdf\" (UniqueName: \"kubernetes.io/projected/c2559280-e517-4a57-8030-8e0592d178cc-kube-api-access-r6tdf\") pod \"network-check-source-7bc5b6bf85-v8shl\" (UID: \"c2559280-e517-4a57-8030-8e0592d178cc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311014 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdw7w\" (UniqueName: \"kubernetes.io/projected/1746da45-43fe-4caf-b8d0-2d945850a2c5-kube-api-access-xdw7w\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311173 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311201 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hbcd\" (UniqueName: \"kubernetes.io/projected/8f904527-8175-4f4c-8a6d-cd33aad0fc65-kube-api-access-8hbcd\") pod \"volume-data-source-validator-78cf4c6f6-gc7kd\" (UID: \"8f904527-8175-4f4c-8a6d-cd33aad0fc65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.311294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311262 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v66fv\" (UniqueName: \"kubernetes.io/projected/3021ee46-d3b1-4152-bec2-24e89cabe038-kube-api-access-v66fv\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.311339 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:19.811313701 +0000 UTC m=+41.165175079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbzz\" (UniqueName: \"kubernetes.io/projected/d7c620db-e75f-4e3e-96c8-2f53474e3357-kube-api-access-lqbzz\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311433 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311530 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.311541 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.311615 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:19.811590027 +0000 UTC m=+41.165451399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311636 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-trusted-ca\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.311671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311666 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-snapshots\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-service-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311728 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7c620db-e75f-4e3e-96c8-2f53474e3357-serving-cert\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311782 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-stats-auth\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311811 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3582e243-6f99-400c-b7e4-e4847ba68e9b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311832 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwb6\" (UniqueName: \"kubernetes.io/projected/3582e243-6f99-400c-b7e4-e4847ba68e9b-kube-api-access-gcwb6\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311857 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-default-certificate\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311874 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.312041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.311881 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313305 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef33513-8b26-482f-84eb-20918d4d1b74-serving-cert\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313391 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-tmp\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7c620db-e75f-4e3e-96c8-2f53474e3357-config\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313389 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3582e243-6f99-400c-b7e4-e4847ba68e9b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313474 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f82b19-2fbb-489f-a569-75d296e7f85c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313519 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcnl6\" (UniqueName: \"kubernetes.io/projected/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-kube-api-access-qcnl6\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313649 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.313788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.313684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.314829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.314725 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f82b19-2fbb-489f-a569-75d296e7f85c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.314829 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.314738 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:19.314829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.314785 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.314829 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.314819 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:19.814796027 +0000 UTC m=+41.168657399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:19.315098 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.314851 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:19.315098 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.314868 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:19.315098 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.314943 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:19.814927914 +0000 UTC m=+41.168789295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:19.317422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.316463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.317422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.316653 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f82b19-2fbb-489f-a569-75d296e7f85c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.317692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.317670 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.319423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.319399 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-default-certificate\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.320093 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.320062 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-stats-auth\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.322410 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.322332 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qj8q\" (UniqueName: \"kubernetes.io/projected/e9f82b19-2fbb-489f-a569-75d296e7f85c-kube-api-access-5qj8q\") pod \"kube-storage-version-migrator-operator-7658d9b749-prkvz\" (UID: \"e9f82b19-2fbb-489f-a569-75d296e7f85c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.322725 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.322701 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mshv\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.322878 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.322830 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hbcd\" (UniqueName: \"kubernetes.io/projected/8f904527-8175-4f4c-8a6d-cd33aad0fc65-kube-api-access-8hbcd\") pod \"volume-data-source-validator-78cf4c6f6-gc7kd\" (UID: \"8f904527-8175-4f4c-8a6d-cd33aad0fc65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" Jul 23 17:05:19.323414 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.323394 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwb6\" (UniqueName: \"kubernetes.io/projected/3582e243-6f99-400c-b7e4-e4847ba68e9b-kube-api-access-gcwb6\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.323504 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.323428 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.323749 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.323732 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v66fv\" (UniqueName: \"kubernetes.io/projected/3021ee46-d3b1-4152-bec2-24e89cabe038-kube-api-access-v66fv\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.414273 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414229 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef33513-8b26-482f-84eb-20918d4d1b74-serving-cert\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.414273 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414275 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-tmp\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.414548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7c620db-e75f-4e3e-96c8-2f53474e3357-config\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.414548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414324 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt4x9\" (UniqueName: \"kubernetes.io/projected/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-kube-api-access-dt4x9\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.414548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414355 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcnl6\" (UniqueName: \"kubernetes.io/projected/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-kube-api-access-qcnl6\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.414548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414452 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.414548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.414757 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414550 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j6vb\" (UniqueName: \"kubernetes.io/projected/1ef33513-8b26-482f-84eb-20918d4d1b74-kube-api-access-2j6vb\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.414757 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414576 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f164089-c9c0-479a-aa51-c2def9952c25-tmp-dir\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.414757 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.414736 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414765 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-config\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.414810 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls podName:1746da45-43fe-4caf-b8d0-2d945850a2c5 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:19.914791068 +0000 UTC m=+41.268652438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hc7cr" (UID: "1746da45-43fe-4caf-b8d0-2d945850a2c5") : secret "samples-operator-tls" not found Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414736 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-tmp\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414831 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-serving-cert\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.414903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414887 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6tdf\" (UniqueName: \"kubernetes.io/projected/c2559280-e517-4a57-8030-8e0592d178cc-kube-api-access-r6tdf\") pod \"network-check-source-7bc5b6bf85-v8shl\" (UID: \"c2559280-e517-4a57-8030-8e0592d178cc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414933 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdw7w\" (UniqueName: \"kubernetes.io/projected/1746da45-43fe-4caf-b8d0-2d945850a2c5-kube-api-access-xdw7w\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.414933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7c620db-e75f-4e3e-96c8-2f53474e3357-config\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbzz\" (UniqueName: \"kubernetes.io/projected/d7c620db-e75f-4e3e-96c8-2f53474e3357-kube-api-access-lqbzz\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-trusted-ca\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415113 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-snapshots\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-service-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7c620db-e75f-4e3e-96c8-2f53474e3357-serving-cert\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.415209 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415189 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpfgf\" (UniqueName: \"kubernetes.io/projected/5f164089-c9c0-479a-aa51-c2def9952c25-kube-api-access-cpfgf\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.415628 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f164089-c9c0-479a-aa51-c2def9952c25-config-volume\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.415628 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415250 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-cert\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.415628 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-config\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.415867 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415841 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-service-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.415960 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415849 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-snapshots\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.415960 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.415934 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.416065 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.416021 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef33513-8b26-482f-84eb-20918d4d1b74-trusted-ca\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.417341 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.417318 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef33513-8b26-482f-84eb-20918d4d1b74-serving-cert\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.417461 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.417379 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-serving-cert\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.417630 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.417613 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d7c620db-e75f-4e3e-96c8-2f53474e3357-serving-cert\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.425675 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.425647 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbzz\" (UniqueName: \"kubernetes.io/projected/d7c620db-e75f-4e3e-96c8-2f53474e3357-kube-api-access-lqbzz\") pod \"service-ca-operator-5bcbc94c78-bnpn2\" (UID: \"d7c620db-e75f-4e3e-96c8-2f53474e3357\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.425844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.425704 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcnl6\" (UniqueName: \"kubernetes.io/projected/5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb-kube-api-access-qcnl6\") pod \"insights-operator-555d9d7885-82hnk\" (UID: \"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb\") " pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.425844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.425731 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6tdf\" (UniqueName: \"kubernetes.io/projected/c2559280-e517-4a57-8030-8e0592d178cc-kube-api-access-r6tdf\") pod \"network-check-source-7bc5b6bf85-v8shl\" (UID: \"c2559280-e517-4a57-8030-8e0592d178cc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" Jul 23 17:05:19.425961 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.425925 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdw7w\" (UniqueName: \"kubernetes.io/projected/1746da45-43fe-4caf-b8d0-2d945850a2c5-kube-api-access-xdw7w\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.429597 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.429581 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" Jul 23 17:05:19.435967 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.435946 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j6vb\" (UniqueName: \"kubernetes.io/projected/1ef33513-8b26-482f-84eb-20918d4d1b74-kube-api-access-2j6vb\") pod \"console-operator-565b7b569d-kfj29\" (UID: \"1ef33513-8b26-482f-84eb-20918d4d1b74\") " pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.456337 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.456316 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" Jul 23 17:05:19.474931 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.474903 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-82hnk" Jul 23 17:05:19.490940 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.490903 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:19.513220 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.513167 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" Jul 23 17:05:19.516372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpfgf\" (UniqueName: \"kubernetes.io/projected/5f164089-c9c0-479a-aa51-c2def9952c25-kube-api-access-cpfgf\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.516372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f164089-c9c0-479a-aa51-c2def9952c25-config-volume\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.516372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516113 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-cert\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.516372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt4x9\" (UniqueName: \"kubernetes.io/projected/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-kube-api-access-dt4x9\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.516372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516208 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.516760 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.516485 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:19.516760 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f164089-c9c0-479a-aa51-c2def9952c25-tmp-dir\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.516760 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.516752 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.016727095 +0000 UTC m=+41.370588458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:19.516947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516800 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f164089-c9c0-479a-aa51-c2def9952c25-config-volume\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.516947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.516942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f164089-c9c0-479a-aa51-c2def9952c25-tmp-dir\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.519431 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.519336 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-cert\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.527084 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.527013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpfgf\" (UniqueName: \"kubernetes.io/projected/5f164089-c9c0-479a-aa51-c2def9952c25-kube-api-access-cpfgf\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:19.528667 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.528216 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt4x9\" (UniqueName: \"kubernetes.io/projected/3ce2a2be-034c-47b2-8a3d-3d38cda5b020-kube-api-access-dt4x9\") pod \"ingress-canary-bzknq\" (UID: \"3ce2a2be-034c-47b2-8a3d-3d38cda5b020\") " pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.534179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.533965 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" Jul 23 17:05:19.590393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.589653 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzknq" Jul 23 17:05:19.699576 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.697190 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz"] Jul 23 17:05:19.699576 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.697300 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd"] Jul 23 17:05:19.717744 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.717693 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f904527_8175_4f4c_8a6d_cd33aad0fc65.slice/crio-1795bac1874f34dd1aa44bdb5070e64269eb3f4fcc206c28e3b6131377191ff7 WatchSource:0}: Error finding container 1795bac1874f34dd1aa44bdb5070e64269eb3f4fcc206c28e3b6131377191ff7: Status 404 returned error can't find the container with id 1795bac1874f34dd1aa44bdb5070e64269eb3f4fcc206c28e3b6131377191ff7 Jul 23 17:05:19.724423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.724320 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-82hnk"] Jul 23 17:05:19.731689 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.731662 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f868cd3_fda3_41e9_aff8_ea7ce09fa0eb.slice/crio-72fa41846096075a1f469feed42dd569c216c6d5b3d6c6438c348508228bb7b7 WatchSource:0}: Error finding container 72fa41846096075a1f469feed42dd569c216c6d5b3d6c6438c348508228bb7b7: Status 404 returned error can't find the container with id 72fa41846096075a1f469feed42dd569c216c6d5b3d6c6438c348508228bb7b7 Jul 23 17:05:19.768030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.767917 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kfj29"] Jul 23 17:05:19.771033 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.770914 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2"] Jul 23 17:05:19.771967 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.771937 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef33513_8b26_482f_84eb_20918d4d1b74.slice/crio-05f896e8153303cd1bbfb52079c2407707b3edc015b4d3a0055a3bda296b154d WatchSource:0}: Error finding container 05f896e8153303cd1bbfb52079c2407707b3edc015b4d3a0055a3bda296b154d: Status 404 returned error can't find the container with id 05f896e8153303cd1bbfb52079c2407707b3edc015b4d3a0055a3bda296b154d Jul 23 17:05:19.773493 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.773462 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c620db_e75f_4e3e_96c8_2f53474e3357.slice/crio-743b26687b6eb571bd539684caf67e636ce2835bfb3fa96d65396a2228b5cfcc WatchSource:0}: Error finding container 743b26687b6eb571bd539684caf67e636ce2835bfb3fa96d65396a2228b5cfcc: Status 404 returned error can't find the container with id 743b26687b6eb571bd539684caf67e636ce2835bfb3fa96d65396a2228b5cfcc Jul 23 17:05:19.786664 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.786618 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl"] Jul 23 17:05:19.791751 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.791726 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2559280_e517_4a57_8030_8e0592d178cc.slice/crio-567bacdd43157eb5863eaf7bc60674e8220f9c7fd79cdfe53c62b13a91c7278e WatchSource:0}: Error finding container 567bacdd43157eb5863eaf7bc60674e8220f9c7fd79cdfe53c62b13a91c7278e: Status 404 returned error can't find the container with id 567bacdd43157eb5863eaf7bc60674e8220f9c7fd79cdfe53c62b13a91c7278e Jul 23 17:05:19.814318 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.814288 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzknq"] Jul 23 17:05:19.817349 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:19.817317 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce2a2be_034c_47b2_8a3d_3d38cda5b020.slice/crio-d857a19e44f77f5721c2c5268270584ad631dafa1fa63fe793593f1be92b0696 WatchSource:0}: Error finding container d857a19e44f77f5721c2c5268270584ad631dafa1fa63fe793593f1be92b0696: Status 404 returned error can't find the container with id d857a19e44f77f5721c2c5268270584ad631dafa1fa63fe793593f1be92b0696 Jul 23 17:05:19.819024 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.819008 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.819101 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.819064 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:19.819243 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.819109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:19.819243 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.819132 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:19.819243 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819152 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.819135257 +0000 UTC m=+42.172996615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:19.819243 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819217 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:19.819243 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819231 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:19.819489 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819216 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:19.819489 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819267 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.819256054 +0000 UTC m=+42.173117438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:19.819489 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819216 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:19.819489 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819294 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.819282415 +0000 UTC m=+42.173143797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:19.819489 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.819314 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.819305391 +0000 UTC m=+42.173166770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:19.920413 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:19.920346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:19.920590 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.920493 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 17:05:19.920590 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:19.920556 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls podName:1746da45-43fe-4caf-b8d0-2d945850a2c5 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:20.920541845 +0000 UTC m=+42.274403202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hc7cr" (UID: "1746da45-43fe-4caf-b8d0-2d945850a2c5") : secret "samples-operator-tls" not found Jul 23 17:05:20.021755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.021649 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:20.022490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.021796 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:20.022490 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.021864 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:21.021847749 +0000 UTC m=+42.375709107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:20.125649 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.125612 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:20.125790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.125612 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:20.129811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.129782 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 17:05:20.129935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.129831 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cgv7q\"" Jul 23 17:05:20.129935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.129830 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 17:05:20.460536 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.460460 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" event={"ID":"e9f82b19-2fbb-489f-a569-75d296e7f85c","Type":"ContainerStarted","Data":"b5bb24438bdf20b29a0fd23d0d40dad048ca44e04b45cfd4897ee04b644e47e4"} Jul 23 17:05:20.462168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.462135 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-82hnk" event={"ID":"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb","Type":"ContainerStarted","Data":"72fa41846096075a1f469feed42dd569c216c6d5b3d6c6438c348508228bb7b7"} Jul 23 17:05:20.464350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.464317 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzknq" event={"ID":"3ce2a2be-034c-47b2-8a3d-3d38cda5b020","Type":"ContainerStarted","Data":"d857a19e44f77f5721c2c5268270584ad631dafa1fa63fe793593f1be92b0696"} Jul 23 17:05:20.466019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.465954 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" event={"ID":"1ef33513-8b26-482f-84eb-20918d4d1b74","Type":"ContainerStarted","Data":"05f896e8153303cd1bbfb52079c2407707b3edc015b4d3a0055a3bda296b154d"} Jul 23 17:05:20.467383 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.467318 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" event={"ID":"8f904527-8175-4f4c-8a6d-cd33aad0fc65","Type":"ContainerStarted","Data":"1795bac1874f34dd1aa44bdb5070e64269eb3f4fcc206c28e3b6131377191ff7"} Jul 23 17:05:20.469104 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.469078 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" event={"ID":"c2559280-e517-4a57-8030-8e0592d178cc","Type":"ContainerStarted","Data":"567bacdd43157eb5863eaf7bc60674e8220f9c7fd79cdfe53c62b13a91c7278e"} Jul 23 17:05:20.471153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.471099 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" event={"ID":"d7c620db-e75f-4e3e-96c8-2f53474e3357","Type":"ContainerStarted","Data":"743b26687b6eb571bd539684caf67e636ce2835bfb3fa96d65396a2228b5cfcc"} Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.830857 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.830912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.831006 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.831081 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831243 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831308 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:22.831289726 +0000 UTC m=+44.185151104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831390 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831423 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:22.831411886 +0000 UTC m=+44.185273258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831495 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831506 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831537 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:22.831526221 +0000 UTC m=+44.185387593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:20.831787 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.831617 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:22.831604331 +0000 UTC m=+44.185465690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:20.932718 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:20.932674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:20.932949 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.932931 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 17:05:20.933025 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:20.933004 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls podName:1746da45-43fe-4caf-b8d0-2d945850a2c5 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:22.932983301 +0000 UTC m=+44.286844681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hc7cr" (UID: "1746da45-43fe-4caf-b8d0-2d945850a2c5") : secret "samples-operator-tls" not found Jul 23 17:05:21.034243 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:21.034205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:21.035058 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:21.034394 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:21.035058 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:21.034471 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:23.034450405 +0000 UTC m=+44.388311787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:22.482230 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.482184 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" event={"ID":"8f904527-8175-4f4c-8a6d-cd33aad0fc65","Type":"ContainerStarted","Data":"42d6137967fdcb88db78ab5229c26be12c90b23c5bd1de2972be0811d0f7a4cb"} Jul 23 17:05:22.502816 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.502443 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-gc7kd" podStartSLOduration=24.19334098 podStartE2EDuration="26.502422061s" totalImagesPullingTime="2.309081081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:22.501453805 +0000 UTC m=+43.855315186" watchObservedRunningTime="2026-07-23 17:05:22.502422061 +0000 UTC m=+43.856283442" Jul 23 17:05:22.618648 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.618605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sl89n_7188a0ac-f5c3-4a35-a37a-c504a176efe0/konnectivity-agent/0.log" limit=1024 Jul 23 17:05:22.752760 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.752669 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sl89n_7188a0ac-f5c3-4a35-a37a-c504a176efe0/konnectivity-agent/0.log" limit=1024 Jul 23 17:05:22.852390 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.852327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:22.852567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.852535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:22.852649 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852618 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:26.852588284 +0000 UTC m=+48.206449648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:22.852711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.852675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:22.852711 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852691 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:22.852711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.852700 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:22.852860 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852746 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:26.852730533 +0000 UTC m=+48.206591924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:22.852860 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852815 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:22.852860 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852832 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:22.852860 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852845 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:22.853020 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852869 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:26.852855079 +0000 UTC m=+48.206716453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:22.853020 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.852890 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:26.852876032 +0000 UTC m=+48.206737389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:22.953298 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:22.953259 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:22.953468 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.953439 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 17:05:22.953526 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:22.953515 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls podName:1746da45-43fe-4caf-b8d0-2d945850a2c5 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:26.953499358 +0000 UTC m=+48.307360716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hc7cr" (UID: "1746da45-43fe-4caf-b8d0-2d945850a2c5") : secret "samples-operator-tls" not found Jul 23 17:05:23.055035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:23.054935 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:23.055186 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:23.055096 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:23.055186 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:23.055175 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:27.055156705 +0000 UTC m=+48.409018065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:26.890334 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:26.889977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.890191 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:34.890163338 +0000 UTC m=+56.244024703 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:26.891029 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:26.891123 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:26.891152 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891183 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891274 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:34.891241205 +0000 UTC m=+56.245102571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891277 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891288 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891302 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891334 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:34.89131675 +0000 UTC m=+56.245178131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:26.891372 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.891352 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:34.891342827 +0000 UTC m=+56.245204191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:26.991827 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:26.991782 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:26.992034 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.991955 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 17:05:26.992096 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:26.992038 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls podName:1746da45-43fe-4caf-b8d0-2d945850a2c5 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:34.992017128 +0000 UTC m=+56.345878492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hc7cr" (UID: "1746da45-43fe-4caf-b8d0-2d945850a2c5") : secret "samples-operator-tls" not found Jul 23 17:05:27.093165 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.093124 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:27.093330 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:27.093304 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:27.093408 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:27.093396 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:35.093375416 +0000 UTC m=+56.447236780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:27.499839 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.499804 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-82hnk" event={"ID":"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb","Type":"ContainerStarted","Data":"eaf782268165c32cb0ea5e1964ec87df34228e9dd61fc24959a7a48a87ff0bd0"} Jul 23 17:05:27.501544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.501439 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzknq" event={"ID":"3ce2a2be-034c-47b2-8a3d-3d38cda5b020","Type":"ContainerStarted","Data":"f15b6847e65cbae096479280310ac343ee7ac0ec4f2ccf43fa981769bceeeff5"} Jul 23 17:05:27.503525 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.503501 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/0.log" Jul 23 17:05:27.503659 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.503545 2584 generic.go:356] "Generic (PLEG): container finished" podID="1ef33513-8b26-482f-84eb-20918d4d1b74" containerID="7635470d7620b6e5b92c465b73c3840b3eaf7f555b9f601af12df16a0df5a788" exitCode=255 Jul 23 17:05:27.503659 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.503634 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" event={"ID":"1ef33513-8b26-482f-84eb-20918d4d1b74","Type":"ContainerDied","Data":"7635470d7620b6e5b92c465b73c3840b3eaf7f555b9f601af12df16a0df5a788"} Jul 23 17:05:27.504074 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.504056 2584 scope.go:117] "RemoveContainer" containerID="7635470d7620b6e5b92c465b73c3840b3eaf7f555b9f601af12df16a0df5a788" Jul 23 17:05:27.506085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.505911 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" event={"ID":"c2559280-e517-4a57-8030-8e0592d178cc","Type":"ContainerStarted","Data":"a5b6e19daee3856c4716eacb03f3fb91af9b5267f54803ae119a6cd346e99054"} Jul 23 17:05:27.508201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.507858 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" event={"ID":"d7c620db-e75f-4e3e-96c8-2f53474e3357","Type":"ContainerStarted","Data":"0db41cc51a4504a623590e5fd4e88cae46399e3bf36800f269e9bf3e04e6dfd2"} Jul 23 17:05:27.509701 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.509664 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" event={"ID":"e9f82b19-2fbb-489f-a569-75d296e7f85c","Type":"ContainerStarted","Data":"e4b85f8f5e952f7982931051f92c224eeaf8b2a142bd9721c41ce2eb5a51005d"} Jul 23 17:05:27.519429 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.518599 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-82hnk" podStartSLOduration=24.867483487 podStartE2EDuration="31.518578208s" totalImagesPullingTime="6.651094721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:27.517784534 +0000 UTC m=+48.871645915" watchObservedRunningTime="2026-07-23 17:05:27.518578208 +0000 UTC m=+48.872439589" Jul 23 17:05:27.554391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.553815 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" podStartSLOduration=22.936893708 podStartE2EDuration="29.553794259s" totalImagesPullingTime="6.616900551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:27.536267792 +0000 UTC m=+48.890129174" watchObservedRunningTime="2026-07-23 17:05:27.553794259 +0000 UTC m=+48.907655643" Jul 23 17:05:27.554565 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.554471 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-v8shl" podStartSLOduration=17.956676948 podStartE2EDuration="24.554461965s" totalImagesPullingTime="6.597785017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:27.552711222 +0000 UTC m=+48.906572603" watchObservedRunningTime="2026-07-23 17:05:27.554461965 +0000 UTC m=+48.908323346" Jul 23 17:05:27.576847 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.576778 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" podStartSLOduration=24.908498507 podStartE2EDuration="31.576741188s" totalImagesPullingTime="6.668242681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:27.575390356 +0000 UTC m=+48.929251731" watchObservedRunningTime="2026-07-23 17:05:27.576741188 +0000 UTC m=+48.930602569" Jul 23 17:05:27.638764 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.638697 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bzknq" podStartSLOduration=2.062041306 podStartE2EDuration="8.638675641s" totalImagesPullingTime="6.576634335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:27.613140116 +0000 UTC m=+48.967001496" watchObservedRunningTime="2026-07-23 17:05:27.638675641 +0000 UTC m=+48.992537022" Jul 23 17:05:27.818403 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.818295 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5gr9t"] Jul 23 17:05:27.821665 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.821636 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:27.842258 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.842226 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 17:05:27.842480 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.842236 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 17:05:27.842638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.842619 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n8wmv\"" Jul 23 17:05:27.845792 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.845742 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5gr9t"] Jul 23 17:05:27.903622 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.903575 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:27.904112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.903754 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:27.904112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.903872 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwctc\" (UniqueName: \"kubernetes.io/projected/57824bfa-1b6c-460f-87f3-a86549527f17-kube-api-access-jwctc\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:27.904112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.903924 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57824bfa-1b6c-460f-87f3-a86549527f17-crio-socket\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:27.904112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:27.903984 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57824bfa-1b6c-460f-87f3-a86549527f17-data-volume\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.004646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004593 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004727 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwctc\" (UniqueName: \"kubernetes.io/projected/57824bfa-1b6c-460f-87f3-a86549527f17-kube-api-access-jwctc\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.004749 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.004830 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:28.504809021 +0000 UTC m=+49.858670399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : configmap "kube-rbac-proxy" not found Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004757 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57824bfa-1b6c-460f-87f3-a86549527f17-crio-socket\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.004877 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.004853 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 17:05:28.005202 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.004915 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:28.504897079 +0000 UTC m=+49.858758440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : secret "insights-runtime-extractor-tls" not found Jul 23 17:05:28.005202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57824bfa-1b6c-460f-87f3-a86549527f17-data-volume\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.005202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.004987 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57824bfa-1b6c-460f-87f3-a86549527f17-crio-socket\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.005383 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.005348 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57824bfa-1b6c-460f-87f3-a86549527f17-data-volume\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.016859 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.016828 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwctc\" (UniqueName: \"kubernetes.io/projected/57824bfa-1b6c-460f-87f3-a86549527f17-kube-api-access-jwctc\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.508752 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.508709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.508942 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.508792 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:28.508942 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.508843 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 17:05:28.508942 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.508915 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:29.508898723 +0000 UTC m=+50.862760096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : configmap "kube-rbac-proxy" not found Jul 23 17:05:28.509055 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.508949 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 17:05:28.509055 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.509008 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:29.508994736 +0000 UTC m=+50.862856097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : secret "insights-runtime-extractor-tls" not found Jul 23 17:05:28.515109 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515084 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:05:28.515493 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515479 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/0.log" Jul 23 17:05:28.515548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515516 2584 generic.go:356] "Generic (PLEG): container finished" podID="1ef33513-8b26-482f-84eb-20918d4d1b74" containerID="8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84" exitCode=255 Jul 23 17:05:28.515652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515630 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" event={"ID":"1ef33513-8b26-482f-84eb-20918d4d1b74","Type":"ContainerDied","Data":"8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84"} Jul 23 17:05:28.515698 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515686 2584 scope.go:117] "RemoveContainer" containerID="7635470d7620b6e5b92c465b73c3840b3eaf7f555b9f601af12df16a0df5a788" Jul 23 17:05:28.515974 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.515955 2584 scope.go:117] "RemoveContainer" containerID="8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84" Jul 23 17:05:28.516166 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:28.516136 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kfj29_openshift-console-operator(1ef33513-8b26-482f-84eb-20918d4d1b74)\"" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" podUID="1ef33513-8b26-482f-84eb-20918d4d1b74" Jul 23 17:05:28.792694 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:28.792613 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hzlv7_cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf/dns-node-resolver/0.log" Jul 23 17:05:29.306712 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.306662 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm"] Jul 23 17:05:29.310320 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.310291 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" Jul 23 17:05:29.313718 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.313692 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 17:05:29.313843 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.313760 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6mt9l\"" Jul 23 17:05:29.334773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.334745 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 17:05:29.385194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.385153 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm"] Jul 23 17:05:29.417319 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.417282 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n6gl\" (UniqueName: \"kubernetes.io/projected/967a098b-fcef-4b09-9dd5-c7bb80868a61-kube-api-access-5n6gl\") pod \"migrator-7cc9c7ffd7-wk6sm\" (UID: \"967a098b-fcef-4b09-9dd5-c7bb80868a61\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" Jul 23 17:05:29.492039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.492002 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:29.492222 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.492057 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:29.519800 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.519764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:29.519800 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.519813 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n6gl\" (UniqueName: \"kubernetes.io/projected/967a098b-fcef-4b09-9dd5-c7bb80868a61-kube-api-access-5n6gl\") pod \"migrator-7cc9c7ffd7-wk6sm\" (UID: \"967a098b-fcef-4b09-9dd5-c7bb80868a61\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" Jul 23 17:05:29.520062 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:29.519884 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 17:05:29.520062 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:29.519944 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:31.51992797 +0000 UTC m=+52.873789331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : configmap "kube-rbac-proxy" not found Jul 23 17:05:29.520062 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.519972 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:29.520265 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:29.520105 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 17:05:29.520265 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:29.520172 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:31.520156213 +0000 UTC m=+52.874017585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : secret "insights-runtime-extractor-tls" not found Jul 23 17:05:29.522288 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.522268 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:05:29.522618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.522604 2584 scope.go:117] "RemoveContainer" containerID="8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84" Jul 23 17:05:29.522768 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:29.522749 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kfj29_openshift-console-operator(1ef33513-8b26-482f-84eb-20918d4d1b74)\"" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" podUID="1ef33513-8b26-482f-84eb-20918d4d1b74" Jul 23 17:05:29.528229 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.528202 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n6gl\" (UniqueName: \"kubernetes.io/projected/967a098b-fcef-4b09-9dd5-c7bb80868a61-kube-api-access-5n6gl\") pod \"migrator-7cc9c7ffd7-wk6sm\" (UID: \"967a098b-fcef-4b09-9dd5-c7bb80868a61\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" Jul 23 17:05:29.620539 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.620436 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" Jul 23 17:05:29.758399 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.758350 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm"] Jul 23 17:05:29.761740 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:29.761708 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967a098b_fcef_4b09_9dd5_c7bb80868a61.slice/crio-1cf0f9366245f8eae74ae8edeb0a266587e790ba105a184e4b0965ebcd49e6ba WatchSource:0}: Error finding container 1cf0f9366245f8eae74ae8edeb0a266587e790ba105a184e4b0965ebcd49e6ba: Status 404 returned error can't find the container with id 1cf0f9366245f8eae74ae8edeb0a266587e790ba105a184e4b0965ebcd49e6ba Jul 23 17:05:29.992107 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:29.992080 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7x68_497a9838-9e96-4f68-8e2a-db3b3172959e/node-ca/0.log" Jul 23 17:05:30.527141 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.527101 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" event={"ID":"967a098b-fcef-4b09-9dd5-c7bb80868a61","Type":"ContainerStarted","Data":"1cf0f9366245f8eae74ae8edeb0a266587e790ba105a184e4b0965ebcd49e6ba"} Jul 23 17:05:30.527557 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.527407 2584 scope.go:117] "RemoveContainer" containerID="8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84" Jul 23 17:05:30.527605 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:30.527570 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kfj29_openshift-console-operator(1ef33513-8b26-482f-84eb-20918d4d1b74)\"" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" podUID="1ef33513-8b26-482f-84eb-20918d4d1b74" Jul 23 17:05:30.591507 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.591475 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzknq_3ce2a2be-034c-47b2-8a3d-3d38cda5b020/serve-healthcheck-canary/0.log" Jul 23 17:05:30.737679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.737638 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fjrn"] Jul 23 17:05:30.740942 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.740915 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.743880 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.743856 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 17:05:30.743880 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.743876 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 17:05:30.744084 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.743926 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 17:05:30.744326 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.744306 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 17:05:30.745002 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.744977 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7gq6l\"" Jul 23 17:05:30.750043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.749963 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fjrn"] Jul 23 17:05:30.832043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.831949 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td9kr\" (UniqueName: \"kubernetes.io/projected/be196290-31e1-48a2-95ce-cefd18bda46c-kube-api-access-td9kr\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.832043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.831992 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be196290-31e1-48a2-95ce-cefd18bda46c-signing-key\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.832043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.832014 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be196290-31e1-48a2-95ce-cefd18bda46c-signing-cabundle\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.933579 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.933518 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td9kr\" (UniqueName: \"kubernetes.io/projected/be196290-31e1-48a2-95ce-cefd18bda46c-kube-api-access-td9kr\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.933757 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.933597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be196290-31e1-48a2-95ce-cefd18bda46c-signing-key\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.933757 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.933626 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be196290-31e1-48a2-95ce-cefd18bda46c-signing-cabundle\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.934404 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.934386 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/be196290-31e1-48a2-95ce-cefd18bda46c-signing-cabundle\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.935985 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.935965 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/be196290-31e1-48a2-95ce-cefd18bda46c-signing-key\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:30.944227 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:30.944198 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td9kr\" (UniqueName: \"kubernetes.io/projected/be196290-31e1-48a2-95ce-cefd18bda46c-kube-api-access-td9kr\") pod \"service-ca-58598695f7-6fjrn\" (UID: \"be196290-31e1-48a2-95ce-cefd18bda46c\") " pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:31.053133 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.053091 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-6fjrn" Jul 23 17:05:31.190250 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.190223 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-6fjrn"] Jul 23 17:05:31.192829 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:31.192791 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe196290_31e1_48a2_95ce_cefd18bda46c.slice/crio-85b57d11d1edc6f222d21bfef5621270cefab834131ad218e9e0c579d96b5807 WatchSource:0}: Error finding container 85b57d11d1edc6f222d21bfef5621270cefab834131ad218e9e0c579d96b5807: Status 404 returned error can't find the container with id 85b57d11d1edc6f222d21bfef5621270cefab834131ad218e9e0c579d96b5807 Jul 23 17:05:31.532025 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.531997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-6fjrn" event={"ID":"be196290-31e1-48a2-95ce-cefd18bda46c","Type":"ContainerStarted","Data":"86596da56ddb0798479dc71d8068226b48a2e71fc32115fb6136aa5d1f4ccd4b"} Jul 23 17:05:31.532508 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.532042 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-6fjrn" event={"ID":"be196290-31e1-48a2-95ce-cefd18bda46c","Type":"ContainerStarted","Data":"85b57d11d1edc6f222d21bfef5621270cefab834131ad218e9e0c579d96b5807"} Jul 23 17:05:31.539207 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.539181 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:31.539328 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.539260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:31.539553 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:31.539421 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 17:05:31.539553 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:31.539516 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 17:05:31.539677 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:31.539538 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:35.539515922 +0000 UTC m=+56.893377298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : configmap "kube-rbac-proxy" not found Jul 23 17:05:31.539677 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:31.539647 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:35.539605331 +0000 UTC m=+56.893466689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : secret "insights-runtime-extractor-tls" not found Jul 23 17:05:31.549415 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:31.549348 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-6fjrn" podStartSLOduration=1.549330586 podStartE2EDuration="1.549330586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:31.548723869 +0000 UTC m=+52.902585250" watchObservedRunningTime="2026-07-23 17:05:31.549330586 +0000 UTC m=+52.903192033" Jul 23 17:05:32.537480 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:32.537439 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" event={"ID":"967a098b-fcef-4b09-9dd5-c7bb80868a61","Type":"ContainerStarted","Data":"381d0d63bb4cc0b03b287939fcb64ebef5adace6ba3ae24679027cab46bfa0bd"} Jul 23 17:05:32.537882 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:32.537503 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" event={"ID":"967a098b-fcef-4b09-9dd5-c7bb80868a61","Type":"ContainerStarted","Data":"9fa7a2efefcf58863ff68bb03ed31be617ef3431729024e38de60765155e78cf"} Jul 23 17:05:32.553822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:32.553771 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-wk6sm" podStartSLOduration=1.796257614 podStartE2EDuration="3.553752721s" totalImagesPullingTime="1.757495107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:32.553538896 +0000 UTC m=+53.907400276" watchObservedRunningTime="2026-07-23 17:05:32.553752721 +0000 UTC m=+53.907614100" Jul 23 17:05:34.972190 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:34.972139 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:34.972246 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972320 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:50.972301689 +0000 UTC m=+72.326163076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : configmap references non-existent config key: service-ca.crt Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:34.972395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:34.972423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972422 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972511 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls podName:3582e243-6f99-400c-b7e4-e4847ba68e9b nodeName:}" failed. No retries permitted until 2026-07-23 17:05:50.972502015 +0000 UTC m=+72.326363373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-k98pv" (UID: "3582e243-6f99-400c-b7e4-e4847ba68e9b") : secret "cluster-monitoring-operator-tls" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972465 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972537 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs podName:3021ee46-d3b1-4152-bec2-24e89cabe038 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:50.972531451 +0000 UTC m=+72.326392808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs") pod "router-default-9f9cf5b5b-kqk57" (UID: "3021ee46-d3b1-4152-bec2-24e89cabe038") : secret "router-metrics-certs-default" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972568 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972590 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f865687dc-g7h52: secret "image-registry-tls" not found Jul 23 17:05:34.972811 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:34.972648 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls podName:feaf45a6-d1e5-482f-a583-a7505d521ff7 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:50.972631522 +0000 UTC m=+72.326492881 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls") pod "image-registry-5f865687dc-g7h52" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7") : secret "image-registry-tls" not found Jul 23 17:05:35.073462 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.073428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:35.075996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.075971 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1746da45-43fe-4caf-b8d0-2d945850a2c5-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hc7cr\" (UID: \"1746da45-43fe-4caf-b8d0-2d945850a2c5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:35.161587 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.161552 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" Jul 23 17:05:35.174949 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.174919 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:35.175089 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.175069 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 17:05:35.175145 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.175136 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls podName:5f164089-c9c0-479a-aa51-c2def9952c25 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:51.175120322 +0000 UTC m=+72.528981680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls") pod "dns-default-p8jvx" (UID: "5f164089-c9c0-479a-aa51-c2def9952c25") : secret "dns-default-metrics-tls" not found Jul 23 17:05:35.330622 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.330588 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr"] Jul 23 17:05:35.549979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.549882 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" event={"ID":"1746da45-43fe-4caf-b8d0-2d945850a2c5","Type":"ContainerStarted","Data":"9753d5534a51784d401c4b3d46e3e25d2ddd5fa7c740cdd029df2bfa14a0acf0"} Jul 23 17:05:35.578496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.578450 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:35.578702 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:35.578521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:35.578702 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.578599 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 17:05:35.578702 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.578628 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 17:05:35.578702 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.578683 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:43.578659073 +0000 UTC m=+64.932520438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : configmap "kube-rbac-proxy" not found Jul 23 17:05:35.578702 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:05:35.578702 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls podName:57824bfa-1b6c-460f-87f3-a86549527f17 nodeName:}" failed. No retries permitted until 2026-07-23 17:05:43.57869303 +0000 UTC m=+64.932554392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gr9t" (UID: "57824bfa-1b6c-460f-87f3-a86549527f17") : secret "insights-runtime-extractor-tls" not found Jul 23 17:05:38.562809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:38.562768 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" event={"ID":"1746da45-43fe-4caf-b8d0-2d945850a2c5","Type":"ContainerStarted","Data":"d71a30dc65db49d36835b938626c548cd06e1bf004d2cef271e21ea3e234d42b"} Jul 23 17:05:38.563227 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:38.562864 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" event={"ID":"1746da45-43fe-4caf-b8d0-2d945850a2c5","Type":"ContainerStarted","Data":"e6c5fc2b83e6d3a895b9f081d88135f9113dea4b9ae2ba5efca2adbec774599c"} Jul 23 17:05:38.582724 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:38.582667 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hc7cr" podStartSLOduration=40.165426544 podStartE2EDuration="42.582650612s" totalImagesPullingTime="2.417224068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:38.580957063 +0000 UTC m=+59.934818444" watchObservedRunningTime="2026-07-23 17:05:38.582650612 +0000 UTC m=+59.936512055" Jul 23 17:05:43.125887 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.125853 2584 scope.go:117] "RemoveContainer" containerID="8afae7d81828164ba4949e98eebf17dc39ed2bb24b4163d6f89435942a08ad84" Jul 23 17:05:43.581039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.581009 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:05:43.581214 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.581075 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" event={"ID":"1ef33513-8b26-482f-84eb-20918d4d1b74","Type":"ContainerStarted","Data":"e14e3e9c86910e211a11f0b01d453c791b10cb1e97294fc89122665c4b3a2391"} Jul 23 17:05:43.581424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.581401 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:43.610704 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.610649 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" podStartSLOduration=40.996693863 podStartE2EDuration="47.610629483s" totalImagesPullingTime="6.61393562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:43.609881437 +0000 UTC m=+64.963742817" watchObservedRunningTime="2026-07-23 17:05:43.610629483 +0000 UTC m=+64.964490866" Jul 23 17:05:43.651185 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.651145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:43.651412 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.651217 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:43.651888 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.651858 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57824bfa-1b6c-460f-87f3-a86549527f17-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:43.653676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.653651 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57824bfa-1b6c-460f-87f3-a86549527f17-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gr9t\" (UID: \"57824bfa-1b6c-460f-87f3-a86549527f17\") " pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:43.735863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.735828 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n8wmv\"" Jul 23 17:05:43.742829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.742801 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5gr9t" Jul 23 17:05:43.894985 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.894956 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-kfj29" Jul 23 17:05:43.911500 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.911476 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5gr9t"] Jul 23 17:05:43.913181 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:43.913155 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57824bfa_1b6c_460f_87f3_a86549527f17.slice/crio-508acfe1f1646cff82e26c9fb0b4c2389953d87c1813f158c42d26e724c077d8 WatchSource:0}: Error finding container 508acfe1f1646cff82e26c9fb0b4c2389953d87c1813f158c42d26e724c077d8: Status 404 returned error can't find the container with id 508acfe1f1646cff82e26c9fb0b4c2389953d87c1813f158c42d26e724c077d8 Jul 23 17:05:43.954842 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.954794 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:43.955036 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.954872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:43.957815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.957792 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 17:05:43.957815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.957804 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 17:05:43.968302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.968272 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/403649e1-c982-4cb0-9868-4b60e570db92-original-pull-secret\") pod \"global-pull-secret-syncer-g42dl\" (UID: \"403649e1-c982-4cb0-9868-4b60e570db92\") " pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:43.968302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:43.968300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7779d10e-5a02-44cf-97e8-300dd7692e76-metrics-certs\") pod \"network-metrics-daemon-9d7qn\" (UID: \"7779d10e-5a02-44cf-97e8-300dd7692e76\") " pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:44.056006 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.055962 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:44.059471 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.059433 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkxdm\" (UniqueName: \"kubernetes.io/projected/64ce3378-8e7f-4b14-a25d-f77329f8e45a-kube-api-access-kkxdm\") pod \"network-check-target-fshkv\" (UID: \"64ce3378-8e7f-4b14-a25d-f77329f8e45a\") " pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:44.064913 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.064879 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fr6xr\"" Jul 23 17:05:44.072525 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.072488 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:44.136541 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.136499 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g42dl" Jul 23 17:05:44.148497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.148464 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cgv7q\"" Jul 23 17:05:44.151772 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.151703 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9d7qn" Jul 23 17:05:44.244021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.243947 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fshkv"] Jul 23 17:05:44.247351 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:44.247315 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ce3378_8e7f_4b14_a25d_f77329f8e45a.slice/crio-f66b1ba550643276c8a2f5f7fbb156c459fbaf1688d2c123ff9afb48e21b9533 WatchSource:0}: Error finding container f66b1ba550643276c8a2f5f7fbb156c459fbaf1688d2c123ff9afb48e21b9533: Status 404 returned error can't find the container with id f66b1ba550643276c8a2f5f7fbb156c459fbaf1688d2c123ff9afb48e21b9533 Jul 23 17:05:44.328048 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.327888 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g42dl"] Jul 23 17:05:44.331054 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:44.331022 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403649e1_c982_4cb0_9868_4b60e570db92.slice/crio-c8a25a9739cd9621286dc80e7a2f029a37348cae1f1808c4650bce66dc1307de WatchSource:0}: Error finding container c8a25a9739cd9621286dc80e7a2f029a37348cae1f1808c4650bce66dc1307de: Status 404 returned error can't find the container with id c8a25a9739cd9621286dc80e7a2f029a37348cae1f1808c4650bce66dc1307de Jul 23 17:05:44.358871 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.358792 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9d7qn"] Jul 23 17:05:44.361714 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:44.361688 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7779d10e_5a02_44cf_97e8_300dd7692e76.slice/crio-3d315d12e8153d385518c66eff1cf7b2f78d75368ac2fd23dc8607b51f700f99 WatchSource:0}: Error finding container 3d315d12e8153d385518c66eff1cf7b2f78d75368ac2fd23dc8607b51f700f99: Status 404 returned error can't find the container with id 3d315d12e8153d385518c66eff1cf7b2f78d75368ac2fd23dc8607b51f700f99 Jul 23 17:05:44.460167 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.460138 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5wgfn" Jul 23 17:05:44.587069 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.587036 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gr9t" event={"ID":"57824bfa-1b6c-460f-87f3-a86549527f17","Type":"ContainerStarted","Data":"508acfe1f1646cff82e26c9fb0b4c2389953d87c1813f158c42d26e724c077d8"} Jul 23 17:05:44.588425 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.588394 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9d7qn" event={"ID":"7779d10e-5a02-44cf-97e8-300dd7692e76","Type":"ContainerStarted","Data":"3d315d12e8153d385518c66eff1cf7b2f78d75368ac2fd23dc8607b51f700f99"} Jul 23 17:05:44.590043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.590008 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g42dl" event={"ID":"403649e1-c982-4cb0-9868-4b60e570db92","Type":"ContainerStarted","Data":"c8a25a9739cd9621286dc80e7a2f029a37348cae1f1808c4650bce66dc1307de"} Jul 23 17:05:44.592069 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.591994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fshkv" event={"ID":"64ce3378-8e7f-4b14-a25d-f77329f8e45a","Type":"ContainerStarted","Data":"e5a698c6aacddb87c68c4ca3659ee4a72d4a90df472f00408b73a9b957b64507"} Jul 23 17:05:44.592069 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.592061 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fshkv" event={"ID":"64ce3378-8e7f-4b14-a25d-f77329f8e45a","Type":"ContainerStarted","Data":"f66b1ba550643276c8a2f5f7fbb156c459fbaf1688d2c123ff9afb48e21b9533"} Jul 23 17:05:44.620881 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:44.620687 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fshkv" podStartSLOduration=65.620664454 podStartE2EDuration="1m5.620664454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:44.619935092 +0000 UTC m=+65.973796484" watchObservedRunningTime="2026-07-23 17:05:44.620664454 +0000 UTC m=+65.974525836" Jul 23 17:05:45.598768 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:45.598732 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:05:51.025896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.025850 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:51.025896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.025895 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:51.026351 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.025945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:51.026351 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.025978 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:51.026689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.026665 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3021ee46-d3b1-4152-bec2-24e89cabe038-service-ca-bundle\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:51.028349 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.028323 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3021ee46-d3b1-4152-bec2-24e89cabe038-metrics-certs\") pod \"router-default-9f9cf5b5b-kqk57\" (UID: \"3021ee46-d3b1-4152-bec2-24e89cabe038\") " pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:51.028616 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.028593 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3582e243-6f99-400c-b7e4-e4847ba68e9b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-k98pv\" (UID: \"3582e243-6f99-400c-b7e4-e4847ba68e9b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:51.028665 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.028593 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"image-registry-5f865687dc-g7h52\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:51.179172 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.179135 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kxsgq\"" Jul 23 17:05:51.185756 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.185728 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" Jul 23 17:05:51.205143 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.205117 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7c5t4\"" Jul 23 17:05:51.213101 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.213059 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:51.218566 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.218531 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wgjfb\"" Jul 23 17:05:51.225513 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.225485 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:51.227212 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.227189 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:51.229498 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.229477 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f164089-c9c0-479a-aa51-c2def9952c25-metrics-tls\") pod \"dns-default-p8jvx\" (UID: \"5f164089-c9c0-479a-aa51-c2def9952c25\") " pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:51.380720 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.380634 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5n9dp\"" Jul 23 17:05:51.387233 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.387205 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:51.883256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.883180 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv"] Jul 23 17:05:51.922036 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.921943 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9f9cf5b5b-kqk57"] Jul 23 17:05:51.928856 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.928829 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p8jvx"] Jul 23 17:05:51.951963 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:51.951933 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:05:52.031325 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:52.031283 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3582e243_6f99_400c_b7e4_e4847ba68e9b.slice/crio-b1c06720fc9b9cfcfe30f9c1cca4f3e87aa9edae24c421b56504d4675d6ddbc6 WatchSource:0}: Error finding container b1c06720fc9b9cfcfe30f9c1cca4f3e87aa9edae24c421b56504d4675d6ddbc6: Status 404 returned error can't find the container with id b1c06720fc9b9cfcfe30f9c1cca4f3e87aa9edae24c421b56504d4675d6ddbc6 Jul 23 17:05:52.032199 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:52.032176 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3021ee46_d3b1_4152_bec2_24e89cabe038.slice/crio-a1fb17ef09312ba2d3de119f92509de85950e462af69642dc1a605583f96fe20 WatchSource:0}: Error finding container a1fb17ef09312ba2d3de119f92509de85950e462af69642dc1a605583f96fe20: Status 404 returned error can't find the container with id a1fb17ef09312ba2d3de119f92509de85950e462af69642dc1a605583f96fe20 Jul 23 17:05:52.033318 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:52.033270 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f164089_c9c0_479a_aa51_c2def9952c25.slice/crio-42b30e61c8ad38217f95ad65485fe4276a53329b10695f91dfa68f9c9fceeb94 WatchSource:0}: Error finding container 42b30e61c8ad38217f95ad65485fe4276a53329b10695f91dfa68f9c9fceeb94: Status 404 returned error can't find the container with id 42b30e61c8ad38217f95ad65485fe4276a53329b10695f91dfa68f9c9fceeb94 Jul 23 17:05:52.034640 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:05:52.034481 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaf45a6_d1e5_482f_a583_a7505d521ff7.slice/crio-3b975d787a9564e925f30e1163721b3b9c480e71c0de20784795d1f68924b96c WatchSource:0}: Error finding container 3b975d787a9564e925f30e1163721b3b9c480e71c0de20784795d1f68924b96c: Status 404 returned error can't find the container with id 3b975d787a9564e925f30e1163721b3b9c480e71c0de20784795d1f68924b96c Jul 23 17:05:52.634044 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.633946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9d7qn" event={"ID":"7779d10e-5a02-44cf-97e8-300dd7692e76","Type":"ContainerStarted","Data":"e8f7acb46dc49bbb90b7cc9d1e08149117daf499e5c5aa79f3db47b56c77fbe0"} Jul 23 17:05:52.634044 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.634023 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9d7qn" event={"ID":"7779d10e-5a02-44cf-97e8-300dd7692e76","Type":"ContainerStarted","Data":"2d4efef3356054b95924bce178521479f53ccec17160d039558bed3c35868e69"} Jul 23 17:05:52.635936 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.635907 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g42dl" event={"ID":"403649e1-c982-4cb0-9868-4b60e570db92","Type":"ContainerStarted","Data":"22339c3dac11c3d0539e5dcdae4b4979785f06890ccb2f752f5b3219af286f93"} Jul 23 17:05:52.637571 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.637546 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8jvx" event={"ID":"5f164089-c9c0-479a-aa51-c2def9952c25","Type":"ContainerStarted","Data":"42b30e61c8ad38217f95ad65485fe4276a53329b10695f91dfa68f9c9fceeb94"} Jul 23 17:05:52.639117 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.639080 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" event={"ID":"3582e243-6f99-400c-b7e4-e4847ba68e9b","Type":"ContainerStarted","Data":"b1c06720fc9b9cfcfe30f9c1cca4f3e87aa9edae24c421b56504d4675d6ddbc6"} Jul 23 17:05:52.640885 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.640862 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" event={"ID":"feaf45a6-d1e5-482f-a583-a7505d521ff7","Type":"ContainerStarted","Data":"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f"} Jul 23 17:05:52.641039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.640911 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:05:52.641039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.640927 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" event={"ID":"feaf45a6-d1e5-482f-a583-a7505d521ff7","Type":"ContainerStarted","Data":"3b975d787a9564e925f30e1163721b3b9c480e71c0de20784795d1f68924b96c"} Jul 23 17:05:52.642649 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.642589 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gr9t" event={"ID":"57824bfa-1b6c-460f-87f3-a86549527f17","Type":"ContainerStarted","Data":"ca0937e656554db9f5233c791df65ae23642934579387d2a34451e92611c5da6"} Jul 23 17:05:52.644162 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.644137 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" event={"ID":"3021ee46-d3b1-4152-bec2-24e89cabe038","Type":"ContainerStarted","Data":"7741cd96b194c19925724fa6540609fda23ff08d9d87363ba2e730d5ff45bc6d"} Jul 23 17:05:52.644269 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.644181 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" event={"ID":"3021ee46-d3b1-4152-bec2-24e89cabe038","Type":"ContainerStarted","Data":"a1fb17ef09312ba2d3de119f92509de85950e462af69642dc1a605583f96fe20"} Jul 23 17:05:52.664888 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.664783 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9d7qn" podStartSLOduration=66.400950353 podStartE2EDuration="1m13.664760827s" totalImagesPullingTime="7.263810474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:52.662707792 +0000 UTC m=+74.016569173" watchObservedRunningTime="2026-07-23 17:05:52.664760827 +0000 UTC m=+74.018622207" Jul 23 17:05:52.682975 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.682924 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g42dl" podStartSLOduration=66.345797626 podStartE2EDuration="1m13.682907178s" totalImagesPullingTime="7.337109552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:52.682452075 +0000 UTC m=+74.036313456" watchObservedRunningTime="2026-07-23 17:05:52.682907178 +0000 UTC m=+74.036768576" Jul 23 17:05:52.709543 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.709490 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" podStartSLOduration=56.70947793 podStartE2EDuration="56.70947793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:52.708929954 +0000 UTC m=+74.062791334" watchObservedRunningTime="2026-07-23 17:05:52.70947793 +0000 UTC m=+74.063339311" Jul 23 17:05:52.736640 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:52.736575 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" podStartSLOduration=73.736547 podStartE2EDuration="1m13.736547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:52.735071382 +0000 UTC m=+74.088932764" watchObservedRunningTime="2026-07-23 17:05:52.736547 +0000 UTC m=+74.090408425" Jul 23 17:05:53.213349 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:53.213308 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:53.216845 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:53.216812 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:53.649802 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:53.649650 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:53.650710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:53.650684 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9f9cf5b5b-kqk57" Jul 23 17:05:57.670425 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.670380 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" event={"ID":"3582e243-6f99-400c-b7e4-e4847ba68e9b","Type":"ContainerStarted","Data":"855ee9e635115728ec6dcdb749163d0c7b7b2b81a01e8c9d72a05f1a5a5e9f94"} Jul 23 17:05:57.672772 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.672735 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gr9t" event={"ID":"57824bfa-1b6c-460f-87f3-a86549527f17","Type":"ContainerStarted","Data":"ca04a57956df76678a890262af838aa851d6b598101f21effd1e79fe1beb0f26"} Jul 23 17:05:57.675007 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.674962 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8jvx" event={"ID":"5f164089-c9c0-479a-aa51-c2def9952c25","Type":"ContainerStarted","Data":"162784fe2ed957d5bf246897e4379d28b825761c291d3d59d6c17acde7a3c137"} Jul 23 17:05:57.675149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.675020 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8jvx" event={"ID":"5f164089-c9c0-479a-aa51-c2def9952c25","Type":"ContainerStarted","Data":"8f6283bedaa23823d707baadc4ef82ca18b7b36803553b2c4a25848fbb7df7c4"} Jul 23 17:05:57.675149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.675132 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p8jvx" Jul 23 17:05:57.699983 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.699905 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-k98pv" podStartSLOduration=57.087390203 podStartE2EDuration="1m1.699882812s" totalImagesPullingTime="4.612492609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:04:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:57.699634848 +0000 UTC m=+79.053496209" watchObservedRunningTime="2026-07-23 17:05:57.699882812 +0000 UTC m=+79.053744193" Jul 23 17:05:57.721844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:57.721786 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p8jvx" podStartSLOduration=34.113229905 podStartE2EDuration="38.721768304s" totalImagesPullingTime="4.608538399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:05:57.71998152 +0000 UTC m=+79.073842901" watchObservedRunningTime="2026-07-23 17:05:57.721768304 +0000 UTC m=+79.075629729" Jul 23 17:05:59.890550 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.890513 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl"] Jul 23 17:05:59.905521 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.905193 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr"] Jul 23 17:05:59.907012 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.906752 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:05:59.911322 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.911131 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 17:05:59.911527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.911399 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c9jjl\"" Jul 23 17:05:59.912973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.912431 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 17:05:59.912973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.912447 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 17:05:59.912973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.912598 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 17:05:59.913235 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.913161 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m"] Jul 23 17:05:59.914919 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.914396 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:05:59.917524 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.917475 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl"] Jul 23 17:05:59.917524 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.917515 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr"] Jul 23 17:05:59.917725 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.917547 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m"] Jul 23 17:05:59.917725 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.917653 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:05:59.917725 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.917681 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 17:05:59.920809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.920591 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 23 17:05:59.920809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.920627 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 23 17:05:59.921196 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.921108 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 23 17:05:59.921335 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.921315 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 23 17:05:59.936838 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.936791 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7"] Jul 23 17:05:59.943259 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.943227 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms"] Jul 23 17:05:59.943689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.943664 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:05:59.952045 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.952007 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gzrpf\"" Jul 23 17:05:59.954410 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.954068 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:05:59.957570 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.957459 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 23 17:05:59.959143 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.959120 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 17:05:59.959466 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.959441 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 17:05:59.959875 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.959800 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-28rgl\"" Jul 23 17:05:59.962469 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.962394 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms"] Jul 23 17:05:59.966430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:05:59.964721 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7"] Jul 23 17:06:00.005278 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005235 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005317 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005421 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v6b8\" (UniqueName: \"kubernetes.io/projected/08cf974b-d929-4d71-b123-6bdccf692be5-kube-api-access-2v6b8\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005466 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fe3863-00fc-4927-9c43-3276fc3cfcbd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08cf974b-d929-4d71-b123-6bdccf692be5-tmp\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005535 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.005568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005559 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08cf974b-d929-4d71-b123-6bdccf692be5-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005604 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsbmv\" (UniqueName: \"kubernetes.io/projected/19fe3863-00fc-4927-9c43-3276fc3cfcbd-kube-api-access-vsbmv\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005693 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005728 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04bdb44a-9080-43e7-81d0-c1d795dc298b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005777 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.005884 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005828 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/434dd052-9b1f-4780-98d5-6a9255044a71-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-4jbv7\" (UID: \"434dd052-9b1f-4780-98d5-6a9255044a71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:00.006211 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.005907 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz5c6\" (UniqueName: \"kubernetes.io/projected/04bdb44a-9080-43e7-81d0-c1d795dc298b-kube-api-access-tz5c6\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.096418 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.096332 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-jkhnd"] Jul 23 17:06:00.103867 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.103836 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:00.104050 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.103987 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:00.107175 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107137 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/434dd052-9b1f-4780-98d5-6a9255044a71-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-4jbv7\" (UID: \"434dd052-9b1f-4780-98d5-6a9255044a71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:00.107340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107241 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz5c6\" (UniqueName: \"kubernetes.io/projected/04bdb44a-9080-43e7-81d0-c1d795dc298b-kube-api-access-tz5c6\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.107340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107266 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107291 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107323 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v6b8\" (UniqueName: \"kubernetes.io/projected/08cf974b-d929-4d71-b123-6bdccf692be5-kube-api-access-2v6b8\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107352 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fe3863-00fc-4927-9c43-3276fc3cfcbd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107394 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08cf974b-d929-4d71-b123-6bdccf692be5-tmp\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107421 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08cf974b-d929-4d71-b123-6bdccf692be5-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbmv\" (UniqueName: \"kubernetes.io/projected/19fe3863-00fc-4927-9c43-3276fc3cfcbd-kube-api-access-vsbmv\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.107627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.107562 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04bdb44a-9080-43e7-81d0-c1d795dc298b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.109868 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.108044 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08cf974b-d929-4d71-b123-6bdccf692be5-tmp\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.109868 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.108728 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.109868 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.109048 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 17:06:00.110619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.110167 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 17:06:00.110619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.110457 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7pwks\"" Jul 23 17:06:00.110619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.110473 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.110807 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.110790 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.111171 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.111140 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/19fe3863-00fc-4927-9c43-3276fc3cfcbd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.112612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.112536 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04bdb44a-9080-43e7-81d0-c1d795dc298b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.113677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.113507 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.113998 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.114226 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zxx5c\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.114078 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115067 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/08cf974b-d929-4d71-b123-6bdccf692be5-klusterlet-config\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115625 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115881 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/434dd052-9b1f-4780-98d5-6a9255044a71-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-4jbv7\" (UID: \"434dd052-9b1f-4780-98d5-6a9255044a71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115895 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115895 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a5a8698a-f0bf-43a4-bb97-5614ba1adc68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-6rdms\" (UID: \"a5a8698a-f0bf-43a4-bb97-5614ba1adc68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.116046 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.115972 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.116636 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.116263 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/19fe3863-00fc-4927-9c43-3276fc3cfcbd-hub\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.119257 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.119188 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-jkhnd"] Jul 23 17:06:00.128022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.127980 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:00.129896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.129832 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz5c6\" (UniqueName: \"kubernetes.io/projected/04bdb44a-9080-43e7-81d0-c1d795dc298b-kube-api-access-tz5c6\") pod \"managed-serviceaccount-addon-agent-777b89cbb4-td6gl\" (UID: \"04bdb44a-9080-43e7-81d0-c1d795dc298b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.130706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.130680 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v6b8\" (UniqueName: \"kubernetes.io/projected/08cf974b-d929-4d71-b123-6bdccf692be5-kube-api-access-2v6b8\") pod \"klusterlet-addon-workmgr-6f79587699-bhmzr\" (UID: \"08cf974b-d929-4d71-b123-6bdccf692be5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.132452 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.132421 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbmv\" (UniqueName: \"kubernetes.io/projected/19fe3863-00fc-4927-9c43-3276fc3cfcbd-kube-api-access-vsbmv\") pod \"cluster-proxy-proxy-agent-64675746c6-8n74m\" (UID: \"19fe3863-00fc-4927-9c43-3276fc3cfcbd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.208782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.208742 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8qkd\" (UniqueName: \"kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.208958 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.208793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.208958 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.208885 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.208958 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.208934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.209115 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.208979 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.209115 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.209072 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgcz7\" (UniqueName: \"kubernetes.io/projected/e2df6662-2c8d-47a9-9457-8a75d33d7579-kube-api-access-rgcz7\") pod \"downloads-5bc67dcd46-jkhnd\" (UID: \"e2df6662-2c8d-47a9-9457-8a75d33d7579\") " pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:00.209199 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.209115 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.233826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.233789 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" Jul 23 17:06:00.246213 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.246178 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:00.259513 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.259480 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" Jul 23 17:06:00.280019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.279983 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:00.287755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.287730 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" Jul 23 17:06:00.310043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310011 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8qkd\" (UniqueName: \"kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310049 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310073 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgcz7\" (UniqueName: \"kubernetes.io/projected/e2df6662-2c8d-47a9-9457-8a75d33d7579-kube-api-access-rgcz7\") pod \"downloads-5bc67dcd46-jkhnd\" (UID: \"e2df6662-2c8d-47a9-9457-8a75d33d7579\") " pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:00.310275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310172 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310860 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310964 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310872 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.310964 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.310882 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.312810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.312781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.312810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.312803 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.322516 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.322486 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8qkd\" (UniqueName: \"kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd\") pod \"console-565d8665cd-c7fxp\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.324126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.324099 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgcz7\" (UniqueName: \"kubernetes.io/projected/e2df6662-2c8d-47a9-9457-8a75d33d7579-kube-api-access-rgcz7\") pod \"downloads-5bc67dcd46-jkhnd\" (UID: \"e2df6662-2c8d-47a9-9457-8a75d33d7579\") " pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:00.454496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.453266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:00.460755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.460636 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:00.565420 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.563056 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m"] Jul 23 17:06:00.583660 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.583584 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl"] Jul 23 17:06:00.587904 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.587728 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04bdb44a_9080_43e7_81d0_c1d795dc298b.slice/crio-1ffc179f54eaa4e61d78a88448112c2d7a375cfc44f2e02849e4db3eb9de810e WatchSource:0}: Error finding container 1ffc179f54eaa4e61d78a88448112c2d7a375cfc44f2e02849e4db3eb9de810e: Status 404 returned error can't find the container with id 1ffc179f54eaa4e61d78a88448112c2d7a375cfc44f2e02849e4db3eb9de810e Jul 23 17:06:00.666750 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.666724 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-jkhnd"] Jul 23 17:06:00.669202 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.669165 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2df6662_2c8d_47a9_9457_8a75d33d7579.slice/crio-84960b27ed802183bd9224cf3c56078ea72eaf71f4098bebb2d3f916f05b45b4 WatchSource:0}: Error finding container 84960b27ed802183bd9224cf3c56078ea72eaf71f4098bebb2d3f916f05b45b4: Status 404 returned error can't find the container with id 84960b27ed802183bd9224cf3c56078ea72eaf71f4098bebb2d3f916f05b45b4 Jul 23 17:06:00.686284 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.686215 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:00.689054 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.689024 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb487f5b_257c_47f6_8f89_14d3ab9f9c34.slice/crio-b79460d7ee03ea918da1346a5ccec6eaed593bff919f79f5721c75250d21a862 WatchSource:0}: Error finding container b79460d7ee03ea918da1346a5ccec6eaed593bff919f79f5721c75250d21a862: Status 404 returned error can't find the container with id b79460d7ee03ea918da1346a5ccec6eaed593bff919f79f5721c75250d21a862 Jul 23 17:06:00.689512 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.689478 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" event={"ID":"04bdb44a-9080-43e7-81d0-c1d795dc298b","Type":"ContainerStarted","Data":"1ffc179f54eaa4e61d78a88448112c2d7a375cfc44f2e02849e4db3eb9de810e"} Jul 23 17:06:00.691782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.691756 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gr9t" event={"ID":"57824bfa-1b6c-460f-87f3-a86549527f17","Type":"ContainerStarted","Data":"bd2d1669ff6b3479eb47c6a5abf7006f0f3deac91f1ba0f75465cdd7c988889e"} Jul 23 17:06:00.693048 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.693020 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" event={"ID":"19fe3863-00fc-4927-9c43-3276fc3cfcbd","Type":"ContainerStarted","Data":"23ef6e1fbedf3cba2fc049f1a0fb511d24252a6a2df335b691aefe3d0402103f"} Jul 23 17:06:00.693959 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.693937 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-jkhnd" event={"ID":"e2df6662-2c8d-47a9-9457-8a75d33d7579","Type":"ContainerStarted","Data":"84960b27ed802183bd9224cf3c56078ea72eaf71f4098bebb2d3f916f05b45b4"} Jul 23 17:06:00.709714 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.709656 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5gr9t" podStartSLOduration=17.481989946 podStartE2EDuration="33.709634626s" totalImagesPullingTime="16.22764468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:05:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:00.708010174 +0000 UTC m=+82.061871545" watchObservedRunningTime="2026-07-23 17:06:00.709634626 +0000 UTC m=+82.063496006" Jul 23 17:06:00.801429 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.801302 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr"] Jul 23 17:06:00.804154 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.804128 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cf974b_d929_4d71_b123_6bdccf692be5.slice/crio-1bd6e9571bde97b2fa035e6be59853480543cdcade0f85bafee35a1e46b6a2cf WatchSource:0}: Error finding container 1bd6e9571bde97b2fa035e6be59853480543cdcade0f85bafee35a1e46b6a2cf: Status 404 returned error can't find the container with id 1bd6e9571bde97b2fa035e6be59853480543cdcade0f85bafee35a1e46b6a2cf Jul 23 17:06:00.808405 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.808377 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7"] Jul 23 17:06:00.811053 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.811030 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434dd052_9b1f_4780_98d5_6a9255044a71.slice/crio-1ca05e45a7fd883d03e153612bf4a73b25d5294e7278fc9e175e46fdf1dac039 WatchSource:0}: Error finding container 1ca05e45a7fd883d03e153612bf4a73b25d5294e7278fc9e175e46fdf1dac039: Status 404 returned error can't find the container with id 1ca05e45a7fd883d03e153612bf4a73b25d5294e7278fc9e175e46fdf1dac039 Jul 23 17:06:00.814593 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:00.814566 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms"] Jul 23 17:06:00.817656 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:00.817628 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a8698a_f0bf_43a4_bb97_5614ba1adc68.slice/crio-fdde3fe319fbec4e007543f4cd4e2b8ab6c7e21957a3c1f09d8c995eb5bcb7b1 WatchSource:0}: Error finding container fdde3fe319fbec4e007543f4cd4e2b8ab6c7e21957a3c1f09d8c995eb5bcb7b1: Status 404 returned error can't find the container with id fdde3fe319fbec4e007543f4cd4e2b8ab6c7e21957a3c1f09d8c995eb5bcb7b1 Jul 23 17:06:01.208076 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.208041 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:06:01.211822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.211779 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.221823 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.221761 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 17:06:01.223135 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.223102 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:06:01.321578 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321535 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321583 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vcm6\" (UniqueName: \"kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321634 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321703 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321729 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.321935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.321807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423184 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423144 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423208 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423234 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vcm6\" (UniqueName: \"kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423323 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423353 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.423430 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.423397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.425923 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.424260 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.425923 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.424841 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.427855 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.426298 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.427855 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.427806 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.433090 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.432897 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.433090 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.433036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.452235 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.452165 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vcm6\" (UniqueName: \"kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6\") pod \"console-f748d87cc-7md6c\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.526168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.526077 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:01.708560 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.708436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" event={"ID":"08cf974b-d929-4d71-b123-6bdccf692be5","Type":"ContainerStarted","Data":"1bd6e9571bde97b2fa035e6be59853480543cdcade0f85bafee35a1e46b6a2cf"} Jul 23 17:06:01.712007 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.711908 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d8665cd-c7fxp" event={"ID":"bb487f5b-257c-47f6-8f89-14d3ab9f9c34","Type":"ContainerStarted","Data":"b79460d7ee03ea918da1346a5ccec6eaed593bff919f79f5721c75250d21a862"} Jul 23 17:06:01.716788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.716692 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" event={"ID":"434dd052-9b1f-4780-98d5-6a9255044a71","Type":"ContainerStarted","Data":"1ca05e45a7fd883d03e153612bf4a73b25d5294e7278fc9e175e46fdf1dac039"} Jul 23 17:06:01.725629 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.725592 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" event={"ID":"a5a8698a-f0bf-43a4-bb97-5614ba1adc68","Type":"ContainerStarted","Data":"fdde3fe319fbec4e007543f4cd4e2b8ab6c7e21957a3c1f09d8c995eb5bcb7b1"} Jul 23 17:06:01.745280 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:01.744265 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:06:02.738157 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:02.738117 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f748d87cc-7md6c" event={"ID":"6452e85a-8c95-4163-bc00-d008a8f0ed9b","Type":"ContainerStarted","Data":"f10d4b8b01900342a6ed5a41235152fb61a7ddb9ee255357bc986152a36a94ed"} Jul 23 17:06:07.682499 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:07.682464 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p8jvx" Jul 23 17:06:10.794475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.794333 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d8665cd-c7fxp" event={"ID":"bb487f5b-257c-47f6-8f89-14d3ab9f9c34","Type":"ContainerStarted","Data":"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7"} Jul 23 17:06:10.798851 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.798772 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" event={"ID":"434dd052-9b1f-4780-98d5-6a9255044a71","Type":"ContainerStarted","Data":"6d8865e358dadcf28a71fef0b480a5cd64760ab043097fbca1292468d719460e"} Jul 23 17:06:10.802175 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.801969 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" event={"ID":"04bdb44a-9080-43e7-81d0-c1d795dc298b","Type":"ContainerStarted","Data":"a1bb84abbbd2c670cff120c7dd3e451167e1410ef4b651b42ff83132844172b0"} Jul 23 17:06:10.809635 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.809342 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" event={"ID":"a5a8698a-f0bf-43a4-bb97-5614ba1adc68","Type":"ContainerStarted","Data":"665bd5f174e85b3cbdd2b594403a859c25a080e333d521eee7f88e1d1a577cb0"} Jul 23 17:06:10.822900 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.821692 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-565d8665cd-c7fxp" podStartSLOduration=0.953378109 podStartE2EDuration="10.821671085s" totalImagesPullingTime="9.868292976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:10.821644857 +0000 UTC m=+92.175506237" watchObservedRunningTime="2026-07-23 17:06:10.821671085 +0000 UTC m=+92.175532466" Jul 23 17:06:10.845435 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:10.845353 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-6rdms" podStartSLOduration=2.114832926 podStartE2EDuration="11.84533078s" totalImagesPullingTime="9.730497854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:10.843444532 +0000 UTC m=+92.197305917" watchObservedRunningTime="2026-07-23 17:06:10.84533078 +0000 UTC m=+92.199192139" Jul 23 17:06:11.232618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.232501 2584 patch_prober.go:28] interesting pod/image-registry-5f865687dc-g7h52 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 17:06:11.232618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.232588 2584 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 17:06:11.816939 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.816897 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" event={"ID":"19fe3863-00fc-4927-9c43-3276fc3cfcbd","Type":"ContainerStarted","Data":"9a0022be2e9f2192edf4a3aaff080053d83f748862d6f98daa9b47ca85ba6699"} Jul 23 17:06:11.817558 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.816953 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" event={"ID":"19fe3863-00fc-4927-9c43-3276fc3cfcbd","Type":"ContainerStarted","Data":"af6d2761d3b97efd46152d298bfaaf267eede442b76ef1b316bd9b91033e0bb3"} Jul 23 17:06:11.817558 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.816972 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" event={"ID":"19fe3863-00fc-4927-9c43-3276fc3cfcbd","Type":"ContainerStarted","Data":"9b790838ae6290bfba138552c82155920f72a9d9fe7d99aca6a2c0f8ea42d013"} Jul 23 17:06:11.819226 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.819192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f748d87cc-7md6c" event={"ID":"6452e85a-8c95-4163-bc00-d008a8f0ed9b","Type":"ContainerStarted","Data":"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3"} Jul 23 17:06:11.820980 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.820937 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" event={"ID":"08cf974b-d929-4d71-b123-6bdccf692be5","Type":"ContainerStarted","Data":"343e2bc8bf0e67217c830e083e2c110e51ae116993f1f36c969a7dec86f344d1"} Jul 23 17:06:11.841342 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.841286 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64675746c6-8n74m" podStartSLOduration=2.848741651 podStartE2EDuration="12.841266458s" totalImagesPullingTime="9.992524807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:11.840686341 +0000 UTC m=+93.194547723" watchObservedRunningTime="2026-07-23 17:06:11.841266458 +0000 UTC m=+93.195127921" Jul 23 17:06:11.842288 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.842237 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-777b89cbb4-td6gl" podStartSLOduration=2.877805134 podStartE2EDuration="12.842224535s" totalImagesPullingTime="9.964419401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:10.866802882 +0000 UTC m=+92.220664360" watchObservedRunningTime="2026-07-23 17:06:11.842224535 +0000 UTC m=+93.196085918" Jul 23 17:06:11.863972 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.863909 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" podStartSLOduration=3.163278608 podStartE2EDuration="12.86389189s" totalImagesPullingTime="9.700613282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:11.862436627 +0000 UTC m=+93.216298007" watchObservedRunningTime="2026-07-23 17:06:11.86389189 +0000 UTC m=+93.217753269" Jul 23 17:06:11.883971 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.883913 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f748d87cc-7md6c" podStartSLOduration=2.074871253 podStartE2EDuration="10.883895653s" totalImagesPullingTime="8.8090244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:11.882017986 +0000 UTC m=+93.235879365" watchObservedRunningTime="2026-07-23 17:06:11.883895653 +0000 UTC m=+93.237757033" Jul 23 17:06:11.900271 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:11.900200 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" podStartSLOduration=3.139100062 podStartE2EDuration="12.900184429s" totalImagesPullingTime="9.761084367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:11.899770696 +0000 UTC m=+93.253632076" watchObservedRunningTime="2026-07-23 17:06:11.900184429 +0000 UTC m=+93.254045810" Jul 23 17:06:12.825708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:12.825676 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:12.826164 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:12.825731 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:12.827711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:12.827654 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6f79587699-bhmzr" Jul 23 17:06:12.831537 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:12.831511 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-4jbv7" Jul 23 17:06:13.654245 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:13.654215 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:06:16.605221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:16.605190 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fshkv" Jul 23 17:06:20.461590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:20.461540 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:20.462086 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:20.461662 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:20.467392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:20.467342 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:20.885915 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:20.885822 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:21.526848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.526808 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:21.528217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.527454 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:21.532688 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.532429 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:21.889080 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.888391 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-jkhnd" event={"ID":"e2df6662-2c8d-47a9-9457-8a75d33d7579","Type":"ContainerStarted","Data":"14b6d423b87f5dbb62abf65759fd4616946a99bea47e46acdb1a87b327692a7a"} Jul 23 17:06:21.889080 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.889009 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:21.890114 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.890088 2584 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-jkhnd container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.26:8080/\": dial tcp 10.132.0.26:8080: connect: connection refused" start-of-body= Jul 23 17:06:21.890245 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.890140 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-jkhnd" podUID="e2df6662-2c8d-47a9-9457-8a75d33d7579" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.26:8080/\": dial tcp 10.132.0.26:8080: connect: connection refused" Jul 23 17:06:21.893856 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.893832 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:06:21.912932 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.912862 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-jkhnd" podStartSLOduration=0.887049977 podStartE2EDuration="21.912837703s" totalImagesPullingTime="21.025787726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:21.910880444 +0000 UTC m=+103.264741825" watchObservedRunningTime="2026-07-23 17:06:21.912837703 +0000 UTC m=+103.266699085" Jul 23 17:06:21.982916 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:21.982877 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:22.018890 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.018856 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h4dzk"] Jul 23 17:06:22.034896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.034866 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.039132 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.039105 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hvq9k\"" Jul 23 17:06:22.039314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.039180 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 17:06:22.039424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.039354 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 17:06:22.039424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.039372 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 17:06:22.039545 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.039512 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 17:06:22.126920 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.126828 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.126920 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.126899 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.126927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-wtmp\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.126955 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdvhq\" (UniqueName: \"kubernetes.io/projected/538386f2-bb08-4d4b-8445-84ca1f37b434-kube-api-access-gdvhq\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.127043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-textfile\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.127069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-sys\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.127093 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-metrics-client-ca\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.127119 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-root\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.127224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.127148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-tls\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228578 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-wtmp\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdvhq\" (UniqueName: \"kubernetes.io/projected/538386f2-bb08-4d4b-8445-84ca1f37b434-kube-api-access-gdvhq\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228770 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-textfile\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228797 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-sys\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-metrics-client-ca\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.228848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228849 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-root\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228862 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-wtmp\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.228883 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-tls\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.229185 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-textfile\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229386 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.229250 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229560 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.229537 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-root\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229653 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.229601 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/538386f2-bb08-4d4b-8445-84ca1f37b434-sys\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.229984 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.229797 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/538386f2-bb08-4d4b-8445-84ca1f37b434-metrics-client-ca\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.231759 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.231736 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-tls\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.232143 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.232118 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/538386f2-bb08-4d4b-8445-84ca1f37b434-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.241603 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.241569 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdvhq\" (UniqueName: \"kubernetes.io/projected/538386f2-bb08-4d4b-8445-84ca1f37b434-kube-api-access-gdvhq\") pod \"node-exporter-h4dzk\" (UID: \"538386f2-bb08-4d4b-8445-84ca1f37b434\") " pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.345475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.345432 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4dzk" Jul 23 17:06:22.377469 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:06:22.377386 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538386f2_bb08_4d4b_8445_84ca1f37b434.slice/crio-d608a9ad853515b0535899a530d3eb7a7f885280fc9171e35bd5bf5102bfabb0 WatchSource:0}: Error finding container d608a9ad853515b0535899a530d3eb7a7f885280fc9171e35bd5bf5102bfabb0: Status 404 returned error can't find the container with id d608a9ad853515b0535899a530d3eb7a7f885280fc9171e35bd5bf5102bfabb0 Jul 23 17:06:22.895951 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.895904 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4dzk" event={"ID":"538386f2-bb08-4d4b-8445-84ca1f37b434","Type":"ContainerStarted","Data":"d608a9ad853515b0535899a530d3eb7a7f885280fc9171e35bd5bf5102bfabb0"} Jul 23 17:06:22.910776 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:22.910741 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-jkhnd" Jul 23 17:06:24.909683 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:24.909593 2584 generic.go:356] "Generic (PLEG): container finished" podID="538386f2-bb08-4d4b-8445-84ca1f37b434" containerID="32c595b45b97c3ad2fd1d500be4d9a21a57dee3e071f9532a8d9ab9837d580a8" exitCode=0 Jul 23 17:06:24.910269 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:24.909692 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4dzk" event={"ID":"538386f2-bb08-4d4b-8445-84ca1f37b434","Type":"ContainerDied","Data":"32c595b45b97c3ad2fd1d500be4d9a21a57dee3e071f9532a8d9ab9837d580a8"} Jul 23 17:06:25.921058 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:25.921020 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4dzk" event={"ID":"538386f2-bb08-4d4b-8445-84ca1f37b434","Type":"ContainerStarted","Data":"cdbf54a2969ead77f80b072b300d25191e60614f04aa7fb67b6e7a9cf8bebad0"} Jul 23 17:06:25.921543 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:25.921086 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4dzk" event={"ID":"538386f2-bb08-4d4b-8445-84ca1f37b434","Type":"ContainerStarted","Data":"9ce681f9702d3f2e8de2f85ab924f8b16ff087af77d1cf2005234b74d7c3b22c"} Jul 23 17:06:25.950853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:25.950791 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h4dzk" podStartSLOduration=2.934388204 podStartE2EDuration="4.95077207s" totalImagesPullingTime="1.908203965s" totalInitContainerRuntime="108.179901ms" isStatefulPod=true podCreationTimestamp="2026-07-23 17:06:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:06:25.947932214 +0000 UTC m=+107.301793608" watchObservedRunningTime="2026-07-23 17:06:25.95077207 +0000 UTC m=+107.304633450" Jul 23 17:06:28.141455 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:28.141414 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:06:37.277876 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:37.277835 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:06:47.921417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:47.921313 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-565d8665cd-c7fxp" podUID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" containerName="console" containerID="cri-o://5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7" gracePeriod=15 Jul 23 17:06:48.027695 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.027644 2584 generic.go:356] "Generic (PLEG): container finished" podID="d7c620db-e75f-4e3e-96c8-2f53474e3357" containerID="0db41cc51a4504a623590e5fd4e88cae46399e3bf36800f269e9bf3e04e6dfd2" exitCode=0 Jul 23 17:06:48.027860 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.027729 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" event={"ID":"d7c620db-e75f-4e3e-96c8-2f53474e3357","Type":"ContainerDied","Data":"0db41cc51a4504a623590e5fd4e88cae46399e3bf36800f269e9bf3e04e6dfd2"} Jul 23 17:06:48.028238 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.028219 2584 scope.go:117] "RemoveContainer" containerID="0db41cc51a4504a623590e5fd4e88cae46399e3bf36800f269e9bf3e04e6dfd2" Jul 23 17:06:48.194022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.193996 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:48.276578 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276541 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.276790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276594 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.276790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276639 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.276790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276684 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.276790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276744 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.276790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.276789 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8qkd\" (UniqueName: \"kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd\") pod \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\" (UID: \"bb487f5b-257c-47f6-8f89-14d3ab9f9c34\") " Jul 23 17:06:48.277073 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.277036 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config" (OuterVolumeSpecName: "console-config") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:06:48.277168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.277122 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:06:48.277168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.277156 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:06:48.279138 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.279106 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd" (OuterVolumeSpecName: "kube-api-access-l8qkd") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "kube-api-access-l8qkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:06:48.279138 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.279129 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:06:48.279275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.279143 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb487f5b-257c-47f6-8f89-14d3ab9f9c34" (UID: "bb487f5b-257c-47f6-8f89-14d3ab9f9c34"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:06:48.377663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377623 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-serving-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:48.377663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377653 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-oauth-config\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:48.377663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377663 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-oauth-serving-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:48.377663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377672 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:48.377933 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377681 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8qkd\" (UniqueName: \"kubernetes.io/projected/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-kube-api-access-l8qkd\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:48.377933 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:48.377689 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb487f5b-257c-47f6-8f89-14d3ab9f9c34-console-config\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:49.033326 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.033294 2584 generic.go:356] "Generic (PLEG): container finished" podID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" containerID="5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7" exitCode=0 Jul 23 17:06:49.033782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.033337 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565d8665cd-c7fxp" Jul 23 17:06:49.033782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.033392 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d8665cd-c7fxp" event={"ID":"bb487f5b-257c-47f6-8f89-14d3ab9f9c34","Type":"ContainerDied","Data":"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7"} Jul 23 17:06:49.033782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.033445 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565d8665cd-c7fxp" event={"ID":"bb487f5b-257c-47f6-8f89-14d3ab9f9c34","Type":"ContainerDied","Data":"b79460d7ee03ea918da1346a5ccec6eaed593bff919f79f5721c75250d21a862"} Jul 23 17:06:49.033782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.033473 2584 scope.go:117] "RemoveContainer" containerID="5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7" Jul 23 17:06:49.035526 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.035503 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-bnpn2" event={"ID":"d7c620db-e75f-4e3e-96c8-2f53474e3357","Type":"ContainerStarted","Data":"cde63a70948a5b5a9f0b4c3e1aafa8a2ffdb80265979d7ed93ed5c77b70cdf5b"} Jul 23 17:06:49.042315 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.042292 2584 scope.go:117] "RemoveContainer" containerID="5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7" Jul 23 17:06:49.042647 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:06:49.042626 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7\": container with ID starting with 5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7 not found: ID does not exist" containerID="5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7" Jul 23 17:06:49.042705 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.042656 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7"} err="failed to get container status \"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7\": rpc error: code = NotFound desc = could not find container \"5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7\": container with ID starting with 5d52979d391ba1e60b22b4ab7c8c3a668a845bcee54ad8fc8fb5a341729d09d7 not found: ID does not exist" Jul 23 17:06:49.081723 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.081682 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:49.088156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.088127 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-565d8665cd-c7fxp"] Jul 23 17:06:49.130638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:49.130595 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" path="/var/lib/kubelet/pods/bb487f5b-257c-47f6-8f89-14d3ab9f9c34/volumes" Jul 23 17:06:52.051205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:52.051169 2584 generic.go:356] "Generic (PLEG): container finished" podID="e9f82b19-2fbb-489f-a569-75d296e7f85c" containerID="e4b85f8f5e952f7982931051f92c224eeaf8b2a142bd9721c41ce2eb5a51005d" exitCode=0 Jul 23 17:06:52.051623 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:52.051215 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" event={"ID":"e9f82b19-2fbb-489f-a569-75d296e7f85c","Type":"ContainerDied","Data":"e4b85f8f5e952f7982931051f92c224eeaf8b2a142bd9721c41ce2eb5a51005d"} Jul 23 17:06:52.051694 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:52.051622 2584 scope.go:117] "RemoveContainer" containerID="e4b85f8f5e952f7982931051f92c224eeaf8b2a142bd9721c41ce2eb5a51005d" Jul 23 17:06:53.056665 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.056632 2584 generic.go:356] "Generic (PLEG): container finished" podID="5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb" containerID="eaf782268165c32cb0ea5e1964ec87df34228e9dd61fc24959a7a48a87ff0bd0" exitCode=0 Jul 23 17:06:53.057095 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.056712 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-82hnk" event={"ID":"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb","Type":"ContainerDied","Data":"eaf782268165c32cb0ea5e1964ec87df34228e9dd61fc24959a7a48a87ff0bd0"} Jul 23 17:06:53.057166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.057141 2584 scope.go:117] "RemoveContainer" containerID="eaf782268165c32cb0ea5e1964ec87df34228e9dd61fc24959a7a48a87ff0bd0" Jul 23 17:06:53.058584 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.058562 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-prkvz" event={"ID":"e9f82b19-2fbb-489f-a569-75d296e7f85c","Type":"ContainerStarted","Data":"81b62a4149c4c6bf6c1f62a5ea31a7e520f444b0b74aa10bb951204dea9a2f4b"} Jul 23 17:06:53.168617 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.168568 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerName="registry" containerID="cri-o://c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f" gracePeriod=30 Jul 23 17:06:53.447542 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.447495 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:06:53.624054 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.623948 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624054 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.623984 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624054 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624011 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624054 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624037 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624066 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624091 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624116 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mshv\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.624587 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624562 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:06:53.624671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.624650 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token\") pod \"feaf45a6-d1e5-482f-a583-a7505d521ff7\" (UID: \"feaf45a6-d1e5-482f-a583-a7505d521ff7\") " Jul 23 17:06:53.625053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.625016 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:06:53.625171 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.625082 2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-certificates\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.627235 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.627205 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:06:53.627235 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.627216 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:06:53.627431 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.627321 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:06:53.627431 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.627385 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv" (OuterVolumeSpecName: "kube-api-access-5mshv") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "kube-api-access-5mshv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:06:53.627513 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.627437 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:06:53.634600 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.634570 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "feaf45a6-d1e5-482f-a583-a7505d521ff7" (UID: "feaf45a6-d1e5-482f-a583-a7505d521ff7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:06:53.725673 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725627 2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-bound-sa-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725673 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725673 2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-installation-pull-secrets\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725689 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/feaf45a6-d1e5-482f-a583-a7505d521ff7-trusted-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725703 2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/feaf45a6-d1e5-482f-a583-a7505d521ff7-ca-trust-extracted\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725718 2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/feaf45a6-d1e5-482f-a583-a7505d521ff7-image-registry-private-configuration\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725734 2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-registry-tls\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:53.725891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:53.725747 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mshv\" (UniqueName: \"kubernetes.io/projected/feaf45a6-d1e5-482f-a583-a7505d521ff7-kube-api-access-5mshv\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:06:54.067162 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.067117 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-82hnk" event={"ID":"5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb","Type":"ContainerStarted","Data":"b7dccdc62fb6ac82efec00d93c69b91ffa5dc542a00a7997efad1e77500fba24"} Jul 23 17:06:54.070166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.069891 2584 generic.go:356] "Generic (PLEG): container finished" podID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerID="c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f" exitCode=0 Jul 23 17:06:54.070166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.069923 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" event={"ID":"feaf45a6-d1e5-482f-a583-a7505d521ff7","Type":"ContainerDied","Data":"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f"} Jul 23 17:06:54.070166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.069961 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" Jul 23 17:06:54.070166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.069977 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f865687dc-g7h52" event={"ID":"feaf45a6-d1e5-482f-a583-a7505d521ff7","Type":"ContainerDied","Data":"3b975d787a9564e925f30e1163721b3b9c480e71c0de20784795d1f68924b96c"} Jul 23 17:06:54.070166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.070006 2584 scope.go:117] "RemoveContainer" containerID="c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f" Jul 23 17:06:54.090255 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.090230 2584 scope.go:117] "RemoveContainer" containerID="c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f" Jul 23 17:06:54.090970 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:06:54.090679 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f\": container with ID starting with c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f not found: ID does not exist" containerID="c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f" Jul 23 17:06:54.090970 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.090721 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f"} err="failed to get container status \"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f\": rpc error: code = NotFound desc = could not find container \"c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f\": container with ID starting with c7dcf8cfdb94fce19c399ea89b571607d606b9c8736d477e835fe0766ecc918f not found: ID does not exist" Jul 23 17:06:54.125869 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.125831 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:06:54.131585 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:54.131557 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f865687dc-g7h52"] Jul 23 17:06:55.130619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:06:55.130583 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" path="/var/lib/kubelet/pods/feaf45a6-d1e5-482f-a583-a7505d521ff7/volumes" Jul 23 17:07:02.299404 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.299310 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f748d87cc-7md6c" podUID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" containerName="console" containerID="cri-o://a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3" gracePeriod=15 Jul 23 17:07:02.583153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.583114 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:07:02.596200 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596173 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596309 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596214 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596309 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596236 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596309 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596269 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596325 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vcm6\" (UniqueName: \"kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596395 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596430 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle\") pod \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\" (UID: \"6452e85a-8c95-4163-bc00-d008a8f0ed9b\") " Jul 23 17:07:02.596899 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596590 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config" (OuterVolumeSpecName: "console-config") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:07:02.596899 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596698 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-config\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.596899 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596812 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:07:02.596899 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596880 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:07:02.597096 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.596895 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:07:02.598606 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.598578 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6" (OuterVolumeSpecName: "kube-api-access-6vcm6") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "kube-api-access-6vcm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:07:02.599004 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.598980 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:07:02.599069 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.599010 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6452e85a-8c95-4163-bc00-d008a8f0ed9b" (UID: "6452e85a-8c95-4163-bc00-d008a8f0ed9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 17:07:02.697463 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697422 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-oauth-serving-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.697463 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697457 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-serving-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.697463 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697467 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6452e85a-8c95-4163-bc00-d008a8f0ed9b-console-oauth-config\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.697708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697476 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vcm6\" (UniqueName: \"kubernetes.io/projected/6452e85a-8c95-4163-bc00-d008a8f0ed9b-kube-api-access-6vcm6\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.697708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697486 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:02.697708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:02.697496 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6452e85a-8c95-4163-bc00-d008a8f0ed9b-trusted-ca-bundle\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:07:03.111469 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.111437 2584 generic.go:356] "Generic (PLEG): container finished" podID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" containerID="a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3" exitCode=0 Jul 23 17:07:03.111684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.111488 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f748d87cc-7md6c" Jul 23 17:07:03.111684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.111524 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f748d87cc-7md6c" event={"ID":"6452e85a-8c95-4163-bc00-d008a8f0ed9b","Type":"ContainerDied","Data":"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3"} Jul 23 17:07:03.111684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.111587 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f748d87cc-7md6c" event={"ID":"6452e85a-8c95-4163-bc00-d008a8f0ed9b","Type":"ContainerDied","Data":"f10d4b8b01900342a6ed5a41235152fb61a7ddb9ee255357bc986152a36a94ed"} Jul 23 17:07:03.111684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.111617 2584 scope.go:117] "RemoveContainer" containerID="a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3" Jul 23 17:07:03.120815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.120797 2584 scope.go:117] "RemoveContainer" containerID="a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3" Jul 23 17:07:03.121103 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:07:03.121079 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3\": container with ID starting with a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3 not found: ID does not exist" containerID="a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3" Jul 23 17:07:03.121168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.121114 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3"} err="failed to get container status \"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3\": rpc error: code = NotFound desc = could not find container \"a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3\": container with ID starting with a859f275aae84cbe1aa37bf30082d445a9431e983c86148584a343a5967968f3 not found: ID does not exist" Jul 23 17:07:03.134640 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.134608 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:07:03.139255 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:03.139227 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f748d87cc-7md6c"] Jul 23 17:07:05.129829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:07:05.129798 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" path="/var/lib/kubelet/pods/6452e85a-8c95-4163-bc00-d008a8f0ed9b/volumes" Jul 23 17:09:39.010912 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:09:39.010879 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:09:39.019083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:09:39.019038 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:09:39.021525 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:09:39.021497 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 17:10:36.339085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.338994 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r"] Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339315 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" containerName="console" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339327 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" containerName="console" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339338 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" containerName="console" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339344 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" containerName="console" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339353 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerName="registry" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339372 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerName="registry" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339427 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb487f5b-257c-47f6-8f89-14d3ab9f9c34" containerName="console" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339437 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="feaf45a6-d1e5-482f-a583-a7505d521ff7" containerName="registry" Jul 23 17:10:36.339518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.339444 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6452e85a-8c95-4163-bc00-d008a8f0ed9b" containerName="console" Jul 23 17:10:36.342495 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.342478 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.353629 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.353601 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 17:10:36.353769 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.353669 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 17:10:36.353769 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.353672 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 17:10:36.353769 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.353720 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 17:10:36.354899 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.354885 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-v7d57\"" Jul 23 17:10:36.357566 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.357548 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.357610 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.357588 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxc8\" (UniqueName: \"kubernetes.io/projected/16087e24-40c7-44f8-8e2d-a683abb54b27-kube-api-access-4sxc8\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.357645 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.357609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-tmp\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.357677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.357641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.357711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.357699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-manifests\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.373337 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.373311 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r"] Jul 23 17:10:36.458876 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.458839 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-manifests\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.459083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.458895 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.459083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.458926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxc8\" (UniqueName: \"kubernetes.io/projected/16087e24-40c7-44f8-8e2d-a683abb54b27-kube-api-access-4sxc8\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.459083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.458948 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-tmp\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.459083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.458973 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.459295 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.459275 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-manifests\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.461139 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.461109 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16087e24-40c7-44f8-8e2d-a683abb54b27-tmp\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.461400 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.461355 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.461453 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.461436 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16087e24-40c7-44f8-8e2d-a683abb54b27-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.477396 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.477346 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxc8\" (UniqueName: \"kubernetes.io/projected/16087e24-40c7-44f8-8e2d-a683abb54b27-kube-api-access-4sxc8\") pod \"opendatahub-operator-controller-manager-f9b469455-qb96r\" (UID: \"16087e24-40c7-44f8-8e2d-a683abb54b27\") " pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.652139 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.652043 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:36.794624 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.794587 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r"] Jul 23 17:10:36.799585 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:10:36.799555 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16087e24_40c7_44f8_8e2d_a683abb54b27.slice/crio-cfd4afdf92406865293d72f35898c4982d85a99a98b376cc50a8b57a9ebb0b9d WatchSource:0}: Error finding container cfd4afdf92406865293d72f35898c4982d85a99a98b376cc50a8b57a9ebb0b9d: Status 404 returned error can't find the container with id cfd4afdf92406865293d72f35898c4982d85a99a98b376cc50a8b57a9ebb0b9d Jul 23 17:10:36.801368 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:36.801340 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 17:10:37.010398 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:37.010345 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" event={"ID":"16087e24-40c7-44f8-8e2d-a683abb54b27","Type":"ContainerStarted","Data":"cfd4afdf92406865293d72f35898c4982d85a99a98b376cc50a8b57a9ebb0b9d"} Jul 23 17:10:41.031990 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:41.031952 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" event={"ID":"16087e24-40c7-44f8-8e2d-a683abb54b27","Type":"ContainerStarted","Data":"b94466e2d1bc6edca6ab633aa1bce96e087aa4caf7a37250fd743e5933923740"} Jul 23 17:10:42.037892 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:42.037855 2584 generic.go:356] "Generic (PLEG): container finished" podID="16087e24-40c7-44f8-8e2d-a683abb54b27" containerID="b94466e2d1bc6edca6ab633aa1bce96e087aa4caf7a37250fd743e5933923740" exitCode=0 Jul 23 17:10:42.038412 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:42.037914 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" event={"ID":"16087e24-40c7-44f8-8e2d-a683abb54b27","Type":"ContainerDied","Data":"b94466e2d1bc6edca6ab633aa1bce96e087aa4caf7a37250fd743e5933923740"} Jul 23 17:10:43.044887 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:43.044843 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" event={"ID":"16087e24-40c7-44f8-8e2d-a683abb54b27","Type":"ContainerStarted","Data":"053549e0667098186bb4ead55a222c044c4c201bcd4a5642b96947dcfea2fdab"} Jul 23 17:10:43.045293 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:43.044987 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:43.075661 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:43.075598 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" podStartSLOduration=2.421377481 podStartE2EDuration="7.075582801s" totalImagesPullingTime="3.55719978s" totalInitContainerRuntime="1.09700554s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:10:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:10:43.073010646 +0000 UTC m=+364.426872037" watchObservedRunningTime="2026-07-23 17:10:43.075582801 +0000 UTC m=+364.429444181" Jul 23 17:10:54.052129 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:54.052094 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f9b469455-qb96r" Jul 23 17:10:56.773425 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.771097 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4"] Jul 23 17:10:56.787265 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.787234 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.791198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.791175 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4dvwc\"" Jul 23 17:10:56.791317 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.791211 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 17:10:56.791317 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.791175 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 17:10:56.799017 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.798991 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4"] Jul 23 17:10:56.831731 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.831698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c79f73f-844e-4588-94b9-77f13a8a2086-tmp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.831895 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.831742 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c79f73f-844e-4588-94b9-77f13a8a2086-tls-certs\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.831895 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.831793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p8zp\" (UniqueName: \"kubernetes.io/projected/2c79f73f-844e-4588-94b9-77f13a8a2086-kube-api-access-7p8zp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.933156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.933115 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c79f73f-844e-4588-94b9-77f13a8a2086-tmp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.933156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.933160 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c79f73f-844e-4588-94b9-77f13a8a2086-tls-certs\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.933417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.933182 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p8zp\" (UniqueName: \"kubernetes.io/projected/2c79f73f-844e-4588-94b9-77f13a8a2086-kube-api-access-7p8zp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.935517 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.935497 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c79f73f-844e-4588-94b9-77f13a8a2086-tmp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.935653 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.935635 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c79f73f-844e-4588-94b9-77f13a8a2086-tls-certs\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:56.983095 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:56.983060 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p8zp\" (UniqueName: \"kubernetes.io/projected/2c79f73f-844e-4588-94b9-77f13a8a2086-kube-api-access-7p8zp\") pod \"kube-auth-proxy-7466c75dcd-bl6q4\" (UID: \"2c79f73f-844e-4588-94b9-77f13a8a2086\") " pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:57.096983 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:57.096886 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" Jul 23 17:10:57.285202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:57.285105 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4"] Jul 23 17:10:57.288049 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:10:57.288024 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c79f73f_844e_4588_94b9_77f13a8a2086.slice/crio-7672d854fdbdb40f3eeee1d6483be22056a24e690c73df456fa7c799c8fa12b4 WatchSource:0}: Error finding container 7672d854fdbdb40f3eeee1d6483be22056a24e690c73df456fa7c799c8fa12b4: Status 404 returned error can't find the container with id 7672d854fdbdb40f3eeee1d6483be22056a24e690c73df456fa7c799c8fa12b4 Jul 23 17:10:58.109701 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.109658 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" event={"ID":"2c79f73f-844e-4588-94b9-77f13a8a2086","Type":"ContainerStarted","Data":"7672d854fdbdb40f3eeee1d6483be22056a24e690c73df456fa7c799c8fa12b4"} Jul 23 17:10:58.405857 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.405778 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f96g6"] Jul 23 17:10:58.411014 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.410992 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.420904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.420883 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 17:10:58.421164 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.421150 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8ztvw\"" Jul 23 17:10:58.441286 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.441224 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f96g6"] Jul 23 17:10:58.548868 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.548834 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180c167e-3fa3-4214-b2e5-74796644b1d5-tls-certs\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.549070 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.548953 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptc5q\" (UniqueName: \"kubernetes.io/projected/180c167e-3fa3-4214-b2e5-74796644b1d5-kube-api-access-ptc5q\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.649667 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.649630 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptc5q\" (UniqueName: \"kubernetes.io/projected/180c167e-3fa3-4214-b2e5-74796644b1d5-kube-api-access-ptc5q\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.649863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.649682 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180c167e-3fa3-4214-b2e5-74796644b1d5-tls-certs\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.652602 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.652562 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180c167e-3fa3-4214-b2e5-74796644b1d5-tls-certs\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.662002 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.661929 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptc5q\" (UniqueName: \"kubernetes.io/projected/180c167e-3fa3-4214-b2e5-74796644b1d5-kube-api-access-ptc5q\") pod \"model-serving-api-54d89bd5fc-f96g6\" (UID: \"180c167e-3fa3-4214-b2e5-74796644b1d5\") " pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.721797 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.721760 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:10:58.896003 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:58.895974 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-f96g6"] Jul 23 17:10:58.898929 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:10:58.898896 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180c167e_3fa3_4214_b2e5_74796644b1d5.slice/crio-8702ef0f000d5e541b3f6b266900d9047a811bd38580589aeb0a51009a69d511 WatchSource:0}: Error finding container 8702ef0f000d5e541b3f6b266900d9047a811bd38580589aeb0a51009a69d511: Status 404 returned error can't find the container with id 8702ef0f000d5e541b3f6b266900d9047a811bd38580589aeb0a51009a69d511 Jul 23 17:10:59.116659 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:10:59.116608 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" event={"ID":"180c167e-3fa3-4214-b2e5-74796644b1d5","Type":"ContainerStarted","Data":"8702ef0f000d5e541b3f6b266900d9047a811bd38580589aeb0a51009a69d511"} Jul 23 17:11:01.132821 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:01.132783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" event={"ID":"2c79f73f-844e-4588-94b9-77f13a8a2086","Type":"ContainerStarted","Data":"02c0eb24be7762e41fdc6fde0c9f485dc16aca50ad0c6cd3b3633a8db7a95469"} Jul 23 17:11:01.164712 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:01.164634 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7466c75dcd-bl6q4" podStartSLOduration=2.303862744 podStartE2EDuration="5.164609268s" totalImagesPullingTime="2.860746524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:11:01.163309431 +0000 UTC m=+382.517170817" watchObservedRunningTime="2026-07-23 17:11:01.164609268 +0000 UTC m=+382.518470649" Jul 23 17:11:02.135870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:02.135836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" event={"ID":"180c167e-3fa3-4214-b2e5-74796644b1d5","Type":"ContainerStarted","Data":"898c79630e5e7836c9264d5dc61b09d0402d8a4d5cd122f9d9d1ad20b98a7592"} Jul 23 17:11:02.136276 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:02.136071 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:11:02.153030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:02.152984 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" podStartSLOduration=1.799324774 podStartE2EDuration="4.152965611s" totalImagesPullingTime="2.353640837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:11:02.152656037 +0000 UTC m=+383.506517416" watchObservedRunningTime="2026-07-23 17:11:02.152965611 +0000 UTC m=+383.506826992" Jul 23 17:11:04.091849 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.091809 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h"] Jul 23 17:11:04.095585 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.095548 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.101504 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.101475 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nznr5\"" Jul 23 17:11:04.101686 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.101665 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 17:11:04.117292 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.117248 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h"] Jul 23 17:11:04.197435 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197445 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197468 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197492 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197575 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197652 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197618 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197864 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197666 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9q45\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-kube-api-access-z9q45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197864 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197728 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.197864 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.197816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132969f0-927a-4c64-b0ba-321a5a51f30d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298432 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298432 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298436 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298697 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298559 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298697 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298601 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298697 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298697 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298696 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9q45\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-kube-api-access-z9q45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298807 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298857 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.298914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.298859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132969f0-927a-4c64-b0ba-321a5a51f30d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.299182 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.299046 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.299182 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.299132 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.299515 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.299490 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.299735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.299713 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132969f0-927a-4c64-b0ba-321a5a51f30d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.300981 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.300962 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.301211 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.301192 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.313401 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.313354 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.313586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.313551 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9q45\" (UniqueName: \"kubernetes.io/projected/132969f0-927a-4c64-b0ba-321a5a51f30d-kube-api-access-z9q45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sw65h\" (UID: \"132969f0-927a-4c64-b0ba-321a5a51f30d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.409613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.409508 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:04.577561 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:04.577523 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h"] Jul 23 17:11:04.586348 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:11:04.586321 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132969f0_927a_4c64_b0ba_321a5a51f30d.slice/crio-162f4aade27a03e9dadf29b9306397bff0c5cb58a26fedfb40113fff0146b860 WatchSource:0}: Error finding container 162f4aade27a03e9dadf29b9306397bff0c5cb58a26fedfb40113fff0146b860: Status 404 returned error can't find the container with id 162f4aade27a03e9dadf29b9306397bff0c5cb58a26fedfb40113fff0146b860 Jul 23 17:11:05.151287 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.151251 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" event={"ID":"132969f0-927a-4c64-b0ba-321a5a51f30d","Type":"ContainerStarted","Data":"162f4aade27a03e9dadf29b9306397bff0c5cb58a26fedfb40113fff0146b860"} Jul 23 17:11:05.252548 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.252514 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp"] Jul 23 17:11:05.259506 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.257636 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.262948 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.262922 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 17:11:05.263129 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.262934 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 17:11:05.263129 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.262985 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6lwtw\"" Jul 23 17:11:05.263129 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.263063 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 17:11:05.281236 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.281200 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp"] Jul 23 17:11:05.309858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.309813 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.309858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.309855 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.310130 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.309875 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.310130 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.309988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvmn\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-kube-api-access-tpvmn\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.310130 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.310036 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.310130 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.310089 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.310308 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.310160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411512 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411567 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411605 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411627 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvmn\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-kube-api-access-tpvmn\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411710 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411701 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.411969 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.411721 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.412495 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.412468 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.414028 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.413996 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.414156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.414052 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.414269 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.414248 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.414387 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.414337 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.420838 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.420809 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.420947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.420881 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvmn\" (UniqueName: \"kubernetes.io/projected/1f8fb21c-fc35-40ae-8ee8-b74f4bae46da-kube-api-access-tpvmn\") pod \"istiod-openshift-gateway-94bb8fbfd-rglvp\" (UID: \"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.570517 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.570479 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:05.766521 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:05.766448 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp"] Jul 23 17:11:05.908776 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:11:05.908734 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8fb21c_fc35_40ae_8ee8_b74f4bae46da.slice/crio-02619ba3c4b93312c3976406bdf6bf251ec26f062517e1d0a81c17170d047fb9 WatchSource:0}: Error finding container 02619ba3c4b93312c3976406bdf6bf251ec26f062517e1d0a81c17170d047fb9: Status 404 returned error can't find the container with id 02619ba3c4b93312c3976406bdf6bf251ec26f062517e1d0a81c17170d047fb9 Jul 23 17:11:06.158806 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:06.158761 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" event={"ID":"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da","Type":"ContainerStarted","Data":"02619ba3c4b93312c3976406bdf6bf251ec26f062517e1d0a81c17170d047fb9"} Jul 23 17:11:09.150636 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:09.150584 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:11:09.150956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:09.150698 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:11:09.150956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:09.150749 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:11:09.152168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:09.152139 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:11:09.152238 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:09.152206 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:11:10.179888 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.179855 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" event={"ID":"132969f0-927a-4c64-b0ba-321a5a51f30d","Type":"ContainerStarted","Data":"671cf5740328107025cb2cf34b8caa5e2b9b8c95585bb178c18b5073c1a94657"} Jul 23 17:11:10.181436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.181412 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" event={"ID":"1f8fb21c-fc35-40ae-8ee8-b74f4bae46da","Type":"ContainerStarted","Data":"db8b577bfe14c3801f74ad95157f847fbd8313d9c27b6956a4aa5255e0388f4f"} Jul 23 17:11:10.181588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.181568 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:10.183071 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.183051 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" Jul 23 17:11:10.204292 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.204240 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" podStartSLOduration=1.642619155 podStartE2EDuration="6.204222186s" totalImagesPullingTime="4.561603031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:11:10.202927903 +0000 UTC m=+391.556789284" watchObservedRunningTime="2026-07-23 17:11:10.204222186 +0000 UTC m=+391.558083631" Jul 23 17:11:10.230083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.230030 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-rglvp" podStartSLOduration=1.989053956 podStartE2EDuration="5.230010675s" totalImagesPullingTime="3.240956719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:11:10.228697425 +0000 UTC m=+391.582558806" watchObservedRunningTime="2026-07-23 17:11:10.230010675 +0000 UTC m=+391.583872054" Jul 23 17:11:10.410436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.410399 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:10.423822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:10.423791 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:11.186829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:11.186791 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:11.187761 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:11.187740 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sw65h" Jul 23 17:11:13.144796 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:11:13.144767 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-f96g6" Jul 23 17:12:44.784945 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.784901 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc"] Jul 23 17:12:44.788576 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.788550 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:44.791971 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.791946 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 17:12:44.793084 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.793062 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 17:12:44.793196 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.793088 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-74j5c\"" Jul 23 17:12:44.804112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.804080 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc"] Jul 23 17:12:44.884557 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.884521 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9rvf\" (UniqueName: \"kubernetes.io/projected/8d8a8d05-4c08-4adf-8a7e-6856df120537-kube-api-access-m9rvf\") pod \"authorino-operator-75cc8d5cb7-dg5xc\" (UID: \"8d8a8d05-4c08-4adf-8a7e-6856df120537\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:44.986009 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:44.985965 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9rvf\" (UniqueName: \"kubernetes.io/projected/8d8a8d05-4c08-4adf-8a7e-6856df120537-kube-api-access-m9rvf\") pod \"authorino-operator-75cc8d5cb7-dg5xc\" (UID: \"8d8a8d05-4c08-4adf-8a7e-6856df120537\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:45.001058 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:45.001018 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9rvf\" (UniqueName: \"kubernetes.io/projected/8d8a8d05-4c08-4adf-8a7e-6856df120537-kube-api-access-m9rvf\") pod \"authorino-operator-75cc8d5cb7-dg5xc\" (UID: \"8d8a8d05-4c08-4adf-8a7e-6856df120537\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:45.099201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:45.099106 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:45.255926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:45.255892 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc"] Jul 23 17:12:45.258982 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:12:45.258945 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8a8d05_4c08_4adf_8a7e_6856df120537.slice/crio-494c81ada7186fce90a6113aebf74ea562cba92b959b87cc4873bc315c8b33c3 WatchSource:0}: Error finding container 494c81ada7186fce90a6113aebf74ea562cba92b959b87cc4873bc315c8b33c3: Status 404 returned error can't find the container with id 494c81ada7186fce90a6113aebf74ea562cba92b959b87cc4873bc315c8b33c3 Jul 23 17:12:45.612991 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:45.612906 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" event={"ID":"8d8a8d05-4c08-4adf-8a7e-6856df120537","Type":"ContainerStarted","Data":"494c81ada7186fce90a6113aebf74ea562cba92b959b87cc4873bc315c8b33c3"} Jul 23 17:12:48.634055 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:48.634012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" event={"ID":"8d8a8d05-4c08-4adf-8a7e-6856df120537","Type":"ContainerStarted","Data":"5f6277c56cd1ee826afaa25ae9402342ea2ca902f67fb2a018c3e262c2a9d3c9"} Jul 23 17:12:48.634533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:48.634158 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:12:48.653329 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:48.653250 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" podStartSLOduration=2.356846497 podStartE2EDuration="4.653229783s" totalImagesPullingTime="2.296383286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:12:48.651344964 +0000 UTC m=+490.005206346" watchObservedRunningTime="2026-07-23 17:12:48.653229783 +0000 UTC m=+490.007091165" Jul 23 17:12:59.640421 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:12:59.640342 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dg5xc" Jul 23 17:13:32.707183 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.707093 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv"] Jul 23 17:13:32.711074 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.711052 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.715098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.715075 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xkgbp\"" Jul 23 17:13:32.731758 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.731719 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv"] Jul 23 17:13:32.808721 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808680 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btgzp\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-kube-api-access-btgzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.808947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808751 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.808947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.808947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808877 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.808947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.809167 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.808958 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13b25b8-9ce2-4354-8629-a744a438f99e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.809167 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.809006 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.809167 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.809063 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.809167 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.809092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.909803 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909762 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909818 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909854 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13b25b8-9ce2-4354-8629-a744a438f99e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909897 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909934 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.909966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910026 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.910007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btgzp\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-kube-api-access-btgzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910330 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.910056 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910330 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.910089 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910466 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.910349 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.910748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.910721 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13b25b8-9ce2-4354-8629-a744a438f99e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.911056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.911008 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.911056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.911032 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.911380 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.911340 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.913987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.913961 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.914105 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.914085 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.922247 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.922220 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:32.922722 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:32.922700 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btgzp\" (UniqueName: \"kubernetes.io/projected/a13b25b8-9ce2-4354-8629-a744a438f99e-kube-api-access-btgzp\") pod \"maas-default-gateway-openshift-default-56654dcf7-rh6gv\" (UID: \"a13b25b8-9ce2-4354-8629-a744a438f99e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:33.022735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.022641 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:33.180900 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.180867 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv"] Jul 23 17:13:33.185094 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:13:33.185063 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13b25b8_9ce2_4354_8629_a744a438f99e.slice/crio-bedbb81da370cd0728ba5c0b5a27270d33b097f4bb5ed36d6f80837671f2386d WatchSource:0}: Error finding container bedbb81da370cd0728ba5c0b5a27270d33b097f4bb5ed36d6f80837671f2386d: Status 404 returned error can't find the container with id bedbb81da370cd0728ba5c0b5a27270d33b097f4bb5ed36d6f80837671f2386d Jul 23 17:13:33.187203 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.187164 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:13:33.187287 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.187253 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:13:33.187328 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.187289 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:13:33.844771 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.844729 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" event={"ID":"a13b25b8-9ce2-4354-8629-a744a438f99e","Type":"ContainerStarted","Data":"0c63962e116b19ae0d7d8b65a3a465c5d4d4744834974719209ea2524c77f107"} Jul 23 17:13:33.845217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.844793 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" event={"ID":"a13b25b8-9ce2-4354-8629-a744a438f99e","Type":"ContainerStarted","Data":"bedbb81da370cd0728ba5c0b5a27270d33b097f4bb5ed36d6f80837671f2386d"} Jul 23 17:13:33.869747 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:33.869689 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" podStartSLOduration=1.869668353 podStartE2EDuration="1.869668353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:13:33.868239172 +0000 UTC m=+535.222100553" watchObservedRunningTime="2026-07-23 17:13:33.869668353 +0000 UTC m=+535.223529733" Jul 23 17:13:34.023174 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:34.023131 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:34.028664 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:34.028638 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:34.849602 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:34.849566 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:13:34.850481 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:13:34.850462 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:14:39.042940 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:39.042908 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:14:39.046343 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:39.046320 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:14:52.818648 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.818610 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-47hhw"] Jul 23 17:14:52.822144 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.822113 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.824809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.824785 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 17:14:52.824955 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.824846 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jznzv\"" Jul 23 17:14:52.832852 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.832819 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-47hhw"] Jul 23 17:14:52.848777 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.848731 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.848972 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.848830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnrmf\" (UniqueName: \"kubernetes.io/projected/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-kube-api-access-fnrmf\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.949521 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.949484 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnrmf\" (UniqueName: \"kubernetes.io/projected/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-kube-api-access-fnrmf\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.949721 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.949556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.950250 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.950231 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:52.958890 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:52.958855 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnrmf\" (UniqueName: \"kubernetes.io/projected/1c4f0d2f-57ca-43f4-93c5-b2229161fde6-kube-api-access-fnrmf\") pod \"payload-pre-processing-7677b774bb-47hhw\" (UID: \"1c4f0d2f-57ca-43f4-93c5-b2229161fde6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:53.135047 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:53.134958 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:53.289392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:53.287345 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-47hhw"] Jul 23 17:14:53.295532 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:14:53.295470 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4f0d2f_57ca_43f4_93c5_b2229161fde6.slice/crio-b284453cec7a0908d4aefacf9716d24d2f2fea8f2835d2a28db03725eeece9c0 WatchSource:0}: Error finding container b284453cec7a0908d4aefacf9716d24d2f2fea8f2835d2a28db03725eeece9c0: Status 404 returned error can't find the container with id b284453cec7a0908d4aefacf9716d24d2f2fea8f2835d2a28db03725eeece9c0 Jul 23 17:14:54.224722 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:54.224627 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" event={"ID":"1c4f0d2f-57ca-43f4-93c5-b2229161fde6","Type":"ContainerStarted","Data":"b284453cec7a0908d4aefacf9716d24d2f2fea8f2835d2a28db03725eeece9c0"} Jul 23 17:14:56.237495 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:56.237457 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" event={"ID":"1c4f0d2f-57ca-43f4-93c5-b2229161fde6","Type":"ContainerStarted","Data":"5d6393703dc673166e3196de9349d755da636194789fb681c97a9771988809a9"} Jul 23 17:14:56.237933 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:56.237582 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:14:56.258323 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:14:56.258264 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" podStartSLOduration=1.6886836650000001 podStartE2EDuration="4.258247584s" totalImagesPullingTime="2.569563919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:14:56.256403321 +0000 UTC m=+617.610264704" watchObservedRunningTime="2026-07-23 17:14:56.258247584 +0000 UTC m=+617.612108962" Jul 23 17:15:00.139391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.139339 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz"] Jul 23 17:15:00.143863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.143838 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" Jul 23 17:15:00.147350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.147311 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:15:00.149882 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.149840 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz"] Jul 23 17:15:00.156630 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.156605 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" Jul 23 17:15:00.312706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:00.311787 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz"] Jul 23 17:15:00.313907 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:15:00.313877 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd1423c_8992_4ce5_a070_3ee67217cc2c.slice/crio-05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0 WatchSource:0}: Error finding container 05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0: Status 404 returned error can't find the container with id 05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0 Jul 23 17:15:01.271274 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.271169 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" event={"ID":"6cd1423c-8992-4ce5-a070-3ee67217cc2c","Type":"ContainerStarted","Data":"05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0"} Jul 23 17:15:01.273044 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.273011 2584 generic.go:356] "Generic (PLEG): container finished" podID="a13b25b8-9ce2-4354-8629-a744a438f99e" containerID="0c63962e116b19ae0d7d8b65a3a465c5d4d4744834974719209ea2524c77f107" exitCode=0 Jul 23 17:15:01.273217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.273085 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" event={"ID":"a13b25b8-9ce2-4354-8629-a744a438f99e","Type":"ContainerDied","Data":"0c63962e116b19ae0d7d8b65a3a465c5d4d4744834974719209ea2524c77f107"} Jul 23 17:15:01.273613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.273594 2584 scope.go:117] "RemoveContainer" containerID="0c63962e116b19ae0d7d8b65a3a465c5d4d4744834974719209ea2524c77f107" Jul 23 17:15:01.274822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.274791 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:15:01.274915 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.274864 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:15:01.274959 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:01.274917 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:15:02.281178 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:02.281131 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" event={"ID":"a13b25b8-9ce2-4354-8629-a744a438f99e","Type":"ContainerStarted","Data":"bb7d546be53c9b4e972f4c9fcf764db636ccb0dacc32fbc81cf4dda91b2fed2c"} Jul 23 17:15:03.023449 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:03.023409 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:15:03.024648 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:03.024625 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rh6gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 17:15:03.024720 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:03.024677 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" podUID="a13b25b8-9ce2-4354-8629-a744a438f99e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 17:15:03.287527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:03.287487 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" event={"ID":"6cd1423c-8992-4ce5-a070-3ee67217cc2c","Type":"ContainerStarted","Data":"b7bcb78c521f1a8c5e0242b71fd4c6d83861417971d81d27ea690d51837ee5cc"} Jul 23 17:15:04.023234 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:04.023193 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rh6gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 17:15:04.023442 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:04.023256 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" podUID="a13b25b8-9ce2-4354-8629-a744a438f99e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 17:15:04.294556 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:04.294422 2584 generic.go:356] "Generic (PLEG): container finished" podID="6cd1423c-8992-4ce5-a070-3ee67217cc2c" containerID="b7bcb78c521f1a8c5e0242b71fd4c6d83861417971d81d27ea690d51837ee5cc" exitCode=0 Jul 23 17:15:04.294556 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:04.294501 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" event={"ID":"6cd1423c-8992-4ce5-a070-3ee67217cc2c","Type":"ContainerDied","Data":"b7bcb78c521f1a8c5e0242b71fd4c6d83861417971d81d27ea690d51837ee5cc"} Jul 23 17:15:04.849697 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:04.849656 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:15:05.023125 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:05.023088 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rh6gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 17:15:05.023301 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:05.023149 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" podUID="a13b25b8-9ce2-4354-8629-a744a438f99e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 17:15:05.469860 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:05.469835 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" Jul 23 17:15:06.023904 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:06.023862 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rh6gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 17:15:06.024081 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:06.023925 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" podUID="a13b25b8-9ce2-4354-8629-a744a438f99e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 17:15:06.313444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:06.310450 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" event={"ID":"6cd1423c-8992-4ce5-a070-3ee67217cc2c","Type":"ContainerDied","Data":"05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0"} Jul 23 17:15:06.313444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:06.310514 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05e9611e35d97b6f098f3eeb81e5ca6db204a95471042d0c85a27646d5c2f3d0" Jul 23 17:15:06.313444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:06.310576 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747115-hs8cz" Jul 23 17:15:07.027681 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:07.027648 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:15:07.028692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:07.028672 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rh6gv" Jul 23 17:15:07.244708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:07.244677 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-47hhw" Jul 23 17:15:09.792733 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.792698 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79"] Jul 23 17:15:09.793111 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.793061 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd1423c-8992-4ce5-a070-3ee67217cc2c" containerName="cleanup" Jul 23 17:15:09.793111 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.793072 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd1423c-8992-4ce5-a070-3ee67217cc2c" containerName="cleanup" Jul 23 17:15:09.793189 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.793136 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd1423c-8992-4ce5-a070-3ee67217cc2c" containerName="cleanup" Jul 23 17:15:09.799605 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.799573 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.802350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.802329 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 17:15:09.802350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.802334 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4mqfb\"" Jul 23 17:15:09.803596 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.803568 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 17:15:09.803823 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.803808 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 17:15:09.805928 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.805908 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79"] Jul 23 17:15:09.920146 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920113 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.920146 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920152 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.920391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920179 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.920391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920197 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9fp\" (UniqueName: \"kubernetes.io/projected/6ce325d2-91bb-4af1-b3a6-30013a90abfe-kube-api-access-gv9fp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.920391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920290 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:09.920391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:09.920316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.021801 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021741 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.021801 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021800 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021821 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9fp\" (UniqueName: \"kubernetes.io/projected/6ce325d2-91bb-4af1-b3a6-30013a90abfe-kube-api-access-gv9fp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021922 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022019 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.021966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022306 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.022287 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022353 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.022327 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.022444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.022412 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.024290 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.024268 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ce325d2-91bb-4af1-b3a6-30013a90abfe-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.024461 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.024444 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ce325d2-91bb-4af1-b3a6-30013a90abfe-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.029713 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.029691 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9fp\" (UniqueName: \"kubernetes.io/projected/6ce325d2-91bb-4af1-b3a6-30013a90abfe-kube-api-access-gv9fp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79\" (UID: \"6ce325d2-91bb-4af1-b3a6-30013a90abfe\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.110823 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.110735 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:10.249523 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.249497 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79"] Jul 23 17:15:10.251515 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:15:10.251481 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce325d2_91bb_4af1_b3a6_30013a90abfe.slice/crio-65a82cedf5ceb7be990154d40e502d3f9f0ae52e301ffe9ab9fe1d5f677a3fc9 WatchSource:0}: Error finding container 65a82cedf5ceb7be990154d40e502d3f9f0ae52e301ffe9ab9fe1d5f677a3fc9: Status 404 returned error can't find the container with id 65a82cedf5ceb7be990154d40e502d3f9f0ae52e301ffe9ab9fe1d5f677a3fc9 Jul 23 17:15:10.330301 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:10.330266 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" event={"ID":"6ce325d2-91bb-4af1-b3a6-30013a90abfe","Type":"ContainerStarted","Data":"65a82cedf5ceb7be990154d40e502d3f9f0ae52e301ffe9ab9fe1d5f677a3fc9"} Jul 23 17:15:12.341203 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:12.341168 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" event={"ID":"6ce325d2-91bb-4af1-b3a6-30013a90abfe","Type":"ContainerStarted","Data":"ace176887192ac9e7e0c8617330ad3f6830ea23aa9d2b1dae6103a221d4f7e12"} Jul 23 17:15:12.362190 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:12.362138 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" podStartSLOduration=2.251423302 podStartE2EDuration="3.362120481s" totalImagesPullingTime="1.110697179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:15:12.361695534 +0000 UTC m=+633.715556926" watchObservedRunningTime="2026-07-23 17:15:12.362120481 +0000 UTC m=+633.715981860" Jul 23 17:15:20.111369 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:20.111324 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:20.111761 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:20.111443 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:20.124433 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:20.124406 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:20.390286 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:20.390206 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79" Jul 23 17:15:22.398384 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.398336 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs"] Jul 23 17:15:22.402112 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.402090 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.405013 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.404988 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 23 17:15:22.414632 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.414606 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs"] Jul 23 17:15:22.441242 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441212 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1e56f08-742b-4e50-8fa8-47adf287e465-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.441443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.441443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441311 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.441443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441332 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.441443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441383 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f45h8\" (UniqueName: \"kubernetes.io/projected/f1e56f08-742b-4e50-8fa8-47adf287e465-kube-api-access-f45h8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.441443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.441431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542470 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f45h8\" (UniqueName: \"kubernetes.io/projected/f1e56f08-742b-4e50-8fa8-47adf287e465-kube-api-access-f45h8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542552 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542612 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1e56f08-742b-4e50-8fa8-47adf287e465-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542881 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542843 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.542930 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542853 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.543006 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.542988 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.544939 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.544914 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f1e56f08-742b-4e50-8fa8-47adf287e465-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.545055 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.545040 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1e56f08-742b-4e50-8fa8-47adf287e465-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.551482 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.551458 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f45h8\" (UniqueName: \"kubernetes.io/projected/f1e56f08-742b-4e50-8fa8-47adf287e465-kube-api-access-f45h8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nxwqs\" (UID: \"f1e56f08-742b-4e50-8fa8-47adf287e465\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.713683 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.713591 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:22.853065 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:22.853036 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs"] Jul 23 17:15:22.855185 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:15:22.855144 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e56f08_742b_4e50_8fa8_47adf287e465.slice/crio-b75520d19ec609637765c45f0581e301fcff0013157026d827080df1caf29a82 WatchSource:0}: Error finding container b75520d19ec609637765c45f0581e301fcff0013157026d827080df1caf29a82: Status 404 returned error can't find the container with id b75520d19ec609637765c45f0581e301fcff0013157026d827080df1caf29a82 Jul 23 17:15:23.395636 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:23.395536 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" event={"ID":"f1e56f08-742b-4e50-8fa8-47adf287e465","Type":"ContainerStarted","Data":"7bfaf9752a912fb2237b0038660a6f620791cda0ba05f08440bc94008457b0e1"} Jul 23 17:15:23.395636 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:23.395598 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" event={"ID":"f1e56f08-742b-4e50-8fa8-47adf287e465","Type":"ContainerStarted","Data":"b75520d19ec609637765c45f0581e301fcff0013157026d827080df1caf29a82"} Jul 23 17:15:23.421700 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:23.421627 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" podStartSLOduration=1.227709256 podStartE2EDuration="1.421607006s" totalImagesPullingTime="193.89775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:15:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:15:23.418811848 +0000 UTC m=+644.772673230" watchObservedRunningTime="2026-07-23 17:15:23.421607006 +0000 UTC m=+644.775468388" Jul 23 17:15:32.714028 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:32.713994 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:32.714569 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:32.714090 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:32.726189 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:32.726167 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:15:33.455225 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:15:33.455188 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nxwqs" Jul 23 17:19:39.081579 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:19:39.081479 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:19:39.083096 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:19:39.083073 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:24:39.113391 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:24:39.113350 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:24:39.114346 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:24:39.114326 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:29:39.148423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:29:39.148398 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:29:39.148901 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:29:39.148737 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:30:00.144977 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.144944 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf"] Jul 23 17:30:00.148282 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.148266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" Jul 23 17:30:00.151038 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.151016 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:30:00.156709 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.156691 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf"] Jul 23 17:30:00.158723 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.158708 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" Jul 23 17:30:00.290847 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.290821 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf"] Jul 23 17:30:00.291940 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:30:00.291916 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89eff329_ccb4_4ea8_82b9_0f1076ee01e2.slice/crio-c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e WatchSource:0}: Error finding container c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e: Status 404 returned error can't find the container with id c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e Jul 23 17:30:00.593885 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.593861 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" event={"ID":"89eff329-ccb4-4ea8-82b9-0f1076ee01e2","Type":"ContainerStarted","Data":"616a1b78945582118336920b29c414e5e550a7e7e84924ff783acbf4a2592251"} Jul 23 17:30:00.594011 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.593932 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" event={"ID":"89eff329-ccb4-4ea8-82b9-0f1076ee01e2","Type":"ContainerStarted","Data":"c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e"} Jul 23 17:30:00.610780 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:00.610740 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" podStartSLOduration=0.610727197 podStartE2EDuration="610.727197ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:30:00.609187422 +0000 UTC m=+1521.963048802" watchObservedRunningTime="2026-07-23 17:30:00.610727197 +0000 UTC m=+1521.964588574" Jul 23 17:30:01.600018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:01.599942 2584 generic.go:356] "Generic (PLEG): container finished" podID="89eff329-ccb4-4ea8-82b9-0f1076ee01e2" containerID="616a1b78945582118336920b29c414e5e550a7e7e84924ff783acbf4a2592251" exitCode=0 Jul 23 17:30:01.600354 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:01.600024 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" event={"ID":"89eff329-ccb4-4ea8-82b9-0f1076ee01e2","Type":"ContainerDied","Data":"616a1b78945582118336920b29c414e5e550a7e7e84924ff783acbf4a2592251"} Jul 23 17:30:02.733903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:02.733882 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" Jul 23 17:30:03.610045 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:03.610021 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" Jul 23 17:30:03.610206 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:03.610062 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747130-68fgf" event={"ID":"89eff329-ccb4-4ea8-82b9-0f1076ee01e2","Type":"ContainerDied","Data":"c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e"} Jul 23 17:30:03.610206 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:30:03.610102 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c081dbf7a02a4bcae4a36d4d857cf74b8899ef4ebcaf75ff41f91bbcf7de680e" Jul 23 17:32:46.633762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.633729 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:32:46.636126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.634211 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89eff329-ccb4-4ea8-82b9-0f1076ee01e2" containerName="cleanup" Jul 23 17:32:46.636126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.634225 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="89eff329-ccb4-4ea8-82b9-0f1076ee01e2" containerName="cleanup" Jul 23 17:32:46.636126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.634331 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="89eff329-ccb4-4ea8-82b9-0f1076ee01e2" containerName="cleanup" Jul 23 17:32:46.637571 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.637549 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.642084 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.642064 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dbb3204a-openshift-default-dockercfg-4hd6z\"" Jul 23 17:32:46.762375 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762331 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762383 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8x6k\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762436 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762479 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762505 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762529 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762555 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.762734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.762655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.774997 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.774966 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:32:46.863419 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863555 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863468 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863555 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8x6k\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863555 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863739 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863563 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863739 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863591 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863739 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863620 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863964 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.863964 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863960 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.864144 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.863996 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.864144 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.864000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.864144 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.864063 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.864305 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.864215 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.864305 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.864237 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.866028 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.866005 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.866261 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.866245 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.893420 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.893374 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8x6k\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.898653 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.898630 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token\") pod \"e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:46.947826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:46.947806 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:47.107785 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.107757 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:32:47.109834 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:32:47.109801 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ffe670_6e31_46c6_b9ad_bf3109c982bc.slice/crio-49e541aba0769080d60c20854b15563873a30edde741bc4713ccf56578f17189 WatchSource:0}: Error finding container 49e541aba0769080d60c20854b15563873a30edde741bc4713ccf56578f17189: Status 404 returned error can't find the container with id 49e541aba0769080d60c20854b15563873a30edde741bc4713ccf56578f17189 Jul 23 17:32:47.111846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.111811 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:32:47.111946 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.111881 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:32:47.111946 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.111909 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:32:47.368933 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.368901 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerStarted","Data":"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28"} Jul 23 17:32:47.369085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.368950 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerStarted","Data":"49e541aba0769080d60c20854b15563873a30edde741bc4713ccf56578f17189"} Jul 23 17:32:47.395829 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.395786 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podStartSLOduration=1.395769178 podStartE2EDuration="1.395769178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:32:47.393892695 +0000 UTC m=+1688.747754075" watchObservedRunningTime="2026-07-23 17:32:47.395769178 +0000 UTC m=+1688.749630626" Jul 23 17:32:47.948506 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:47.948478 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:48.952853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:48.952818 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:49.378894 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:49.378810 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:32:49.379718 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:32:49.379701 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:04.380039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.380013 2584 patch_prober.go:28] interesting pod/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 17:33:04.380437 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.380058 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 17:33:04.453972 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.453949 2584 generic.go:356] "Generic (PLEG): container finished" podID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerID="dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28" exitCode=0 Jul 23 17:33:04.454064 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.454021 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerDied","Data":"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28"} Jul 23 17:33:04.454421 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.454405 2584 scope.go:117] "RemoveContainer" containerID="dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28" Jul 23 17:33:04.455247 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.455220 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:04.455303 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.455290 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:04.455337 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:04.455324 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:05.460773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:05.460744 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerStarted","Data":"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf"} Jul 23 17:33:05.948822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:05.948785 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:05.949020 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:05.948997 2584 patch_prober.go:28] interesting pod/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 17:33:05.949068 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:05.949047 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 17:33:06.949085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:06.949050 2584 patch_prober.go:28] interesting pod/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 17:33:06.949477 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:06.949114 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 17:33:07.948689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:07.948650 2584 patch_prober.go:28] interesting pod/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 17:33:07.948889 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:07.948699 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 17:33:08.949099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:08.949067 2584 patch_prober.go:28] interesting pod/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 17:33:08.949482 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:08.949116 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 17:33:09.951815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:09.951785 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:09.952185 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:09.952143 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:09.952922 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:09.952906 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:14.516228 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.516188 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f"] Jul 23 17:33:14.519752 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.519729 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:14.523687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.523668 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:33:14.524074 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.524059 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 17:33:14.538275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.538249 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f"] Jul 23 17:33:14.586629 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.586607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f\" (UID: \"f0fd3738-7cde-4b64-ac29-00f92534f0f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:14.687276 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.687253 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f\" (UID: \"f0fd3738-7cde-4b64-ac29-00f92534f0f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:14.687885 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.687862 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f\" (UID: \"f0fd3738-7cde-4b64-ac29-00f92534f0f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:14.829614 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:14.829558 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:15.165469 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:15.165448 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f"] Jul 23 17:33:15.167184 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:33:15.167155 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fd3738_7cde_4b64_ac29_00f92534f0f3.slice/crio-d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f WatchSource:0}: Error finding container d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f: Status 404 returned error can't find the container with id d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f Jul 23 17:33:15.508290 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:15.508259 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" event={"ID":"f0fd3738-7cde-4b64-ac29-00f92534f0f3","Type":"ContainerStarted","Data":"92fe1e084fc98ac06daf1c4781f853ed3c86411458c6b7f057da1e7f42586140"} Jul 23 17:33:15.508434 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:15.508308 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" event={"ID":"f0fd3738-7cde-4b64-ac29-00f92534f0f3","Type":"ContainerStarted","Data":"d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f"} Jul 23 17:33:15.531845 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:15.531803 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" podStartSLOduration=1.531787646 podStartE2EDuration="1.531787646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:33:15.529120731 +0000 UTC m=+1716.882982112" watchObservedRunningTime="2026-07-23 17:33:15.531787646 +0000 UTC m=+1716.885649025" Jul 23 17:33:16.514510 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:16.514484 2584 generic.go:356] "Generic (PLEG): container finished" podID="f0fd3738-7cde-4b64-ac29-00f92534f0f3" containerID="92fe1e084fc98ac06daf1c4781f853ed3c86411458c6b7f057da1e7f42586140" exitCode=0 Jul 23 17:33:16.514624 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:16.514568 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" event={"ID":"f0fd3738-7cde-4b64-ac29-00f92534f0f3","Type":"ContainerDied","Data":"92fe1e084fc98ac06daf1c4781f853ed3c86411458c6b7f057da1e7f42586140"} Jul 23 17:33:17.656203 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:17.656180 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:17.710353 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:17.710330 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca\") pod \"f0fd3738-7cde-4b64-ac29-00f92534f0f3\" (UID: \"f0fd3738-7cde-4b64-ac29-00f92534f0f3\") " Jul 23 17:33:17.710645 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:17.710623 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0fd3738-7cde-4b64-ac29-00f92534f0f3" (UID: "f0fd3738-7cde-4b64-ac29-00f92534f0f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:33:17.810995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:17.810947 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0fd3738-7cde-4b64-ac29-00f92534f0f3-maas-api-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:18.525976 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:18.525957 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" Jul 23 17:33:18.525976 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:18.525962 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f" event={"ID":"f0fd3738-7cde-4b64-ac29-00f92534f0f3","Type":"ContainerDied","Data":"d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f"} Jul 23 17:33:18.526133 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:18.525998 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9c06538c9251e18021f3ab7f944d7ef4c20c7210198204b339d95d003b6190f" Jul 23 17:33:29.597375 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:29.597330 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f"] Jul 23 17:33:29.600171 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:29.600144 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dbb3204a-2vs6f"] Jul 23 17:33:29.743557 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:29.743528 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:33:29.743801 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:29.743774 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" containerID="cri-o://e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf" gracePeriod=30 Jul 23 17:33:30.843571 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.843536 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:33:30.844031 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.843988 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0fd3738-7cde-4b64-ac29-00f92534f0f3" containerName="revoke-keys" Jul 23 17:33:30.844031 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.844007 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0fd3738-7cde-4b64-ac29-00f92534f0f3" containerName="revoke-keys" Jul 23 17:33:30.844147 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.844122 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0fd3738-7cde-4b64-ac29-00f92534f0f3" containerName="revoke-keys" Jul 23 17:33:30.847474 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.847454 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.850063 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.850044 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ab1734b0-gw-openshift-default-dockercfg-4ndsg\"" Jul 23 17:33:30.861419 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.861393 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:33:30.917437 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917450 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917479 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917549 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd8mf\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917797 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917797 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917728 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917898 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917804 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:30.917898 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:30.917839 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018627 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018595 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018639 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018680 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018714 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.018826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019088 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018911 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd8mf\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019088 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.018955 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019088 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019009 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019118 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019230 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019354 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019341 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019552 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019525 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.019552 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.019536 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.021397 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.021352 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.021549 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.021529 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.027643 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.027609 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd8mf\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.027817 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.027799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token\") pod \"e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.132459 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.132394 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0fd3738-7cde-4b64-ac29-00f92534f0f3" path="/var/lib/kubelet/pods/f0fd3738-7cde-4b64-ac29-00f92534f0f3/volumes" Jul 23 17:33:31.160549 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.160530 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:31.311776 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.311741 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:33:31.315216 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:33:31.315186 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab39f0c1_fb36_4ce6_ad0b_11b4a20749e6.slice/crio-e3bb444a0570c4e3d97692c7647afc4a9789a1413c6ec3030a073bc767633fbd WatchSource:0}: Error finding container e3bb444a0570c4e3d97692c7647afc4a9789a1413c6ec3030a073bc767633fbd: Status 404 returned error can't find the container with id e3bb444a0570c4e3d97692c7647afc4a9789a1413c6ec3030a073bc767633fbd Jul 23 17:33:31.317598 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.317560 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:31.317704 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.317634 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:31.317704 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.317663 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:31.593465 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.593425 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerStarted","Data":"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e"} Jul 23 17:33:31.593638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.593484 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerStarted","Data":"e3bb444a0570c4e3d97692c7647afc4a9789a1413c6ec3030a073bc767633fbd"} Jul 23 17:33:31.618110 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:31.618054 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podStartSLOduration=1.618038988 podStartE2EDuration="1.618038988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:33:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:33:31.615230782 +0000 UTC m=+1732.969092162" watchObservedRunningTime="2026-07-23 17:33:31.618038988 +0000 UTC m=+1732.971900368" Jul 23 17:33:32.161674 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:32.161643 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:32.166543 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:32.166521 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:32.598939 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:32.598912 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:32.599999 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:32.599978 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:34.993701 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:34.993678 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:35.055295 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055237 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055295 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055273 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055323 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055387 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055441 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055489 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8x6k\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055700 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055516 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055700 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055540 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055700 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055579 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket\") pod \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\" (UID: \"c8ffe670-6e31-46c6-b9ad-bf3109c982bc\") " Jul 23 17:33:35.055826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055742 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:33:35.055826 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055799 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:33:35.056002 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.055983 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.056066 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.056009 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.056066 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.056015 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:33:35.056208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.056189 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:33:35.056345 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.056321 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:33:35.057671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.057644 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:33:35.057761 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.057748 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k" (OuterVolumeSpecName: "kube-api-access-j8x6k") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "kube-api-access-j8x6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:33:35.058021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.058002 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:33:35.058123 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.058105 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8ffe670-6e31-46c6-b9ad-bf3109c982bc" (UID: "c8ffe670-6e31-46c6-b9ad-bf3109c982bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:33:35.156911 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156888 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.156911 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156912 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.157056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156921 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.157056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156929 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.157056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156937 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.157056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156945 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8x6k\" (UniqueName: \"kubernetes.io/projected/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-kube-api-access-j8x6k\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.157056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.156954 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ffe670-6e31-46c6-b9ad-bf3109c982bc-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:33:35.614618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.614585 2584 generic.go:356] "Generic (PLEG): container finished" podID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerID="e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf" exitCode=0 Jul 23 17:33:35.614778 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.614629 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerDied","Data":"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf"} Jul 23 17:33:35.614778 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.614637 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" Jul 23 17:33:35.614778 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.614682 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l" event={"ID":"c8ffe670-6e31-46c6-b9ad-bf3109c982bc","Type":"ContainerDied","Data":"49e541aba0769080d60c20854b15563873a30edde741bc4713ccf56578f17189"} Jul 23 17:33:35.614778 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.614713 2584 scope.go:117] "RemoveContainer" containerID="e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf" Jul 23 17:33:35.623741 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.623722 2584 scope.go:117] "RemoveContainer" containerID="dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28" Jul 23 17:33:35.631996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.631977 2584 scope.go:117] "RemoveContainer" containerID="e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf" Jul 23 17:33:35.632285 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:33:35.632265 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf\": container with ID starting with e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf not found: ID does not exist" containerID="e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf" Jul 23 17:33:35.632385 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.632292 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf"} err="failed to get container status \"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf\": rpc error: code = NotFound desc = could not find container \"e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf\": container with ID starting with e1b521aecc428a6331b42f0264b938e785fbf1f6ed555459734f7c911a3577bf not found: ID does not exist" Jul 23 17:33:35.632385 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.632310 2584 scope.go:117] "RemoveContainer" containerID="dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28" Jul 23 17:33:35.632579 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:33:35.632562 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28\": container with ID starting with dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28 not found: ID does not exist" containerID="dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28" Jul 23 17:33:35.632638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.632583 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28"} err="failed to get container status \"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28\": rpc error: code = NotFound desc = could not find container \"dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28\": container with ID starting with dd0b524a37f3e93f8684db668ff34744ccd808cf8932708f96d0f6cb94253a28 not found: ID does not exist" Jul 23 17:33:35.634053 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.634031 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:33:35.640075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:35.640054 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dbb3204a-openshift-default-b67697ccf-c9q8l"] Jul 23 17:33:37.131774 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:37.131739 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" path="/var/lib/kubelet/pods/c8ffe670-6e31-46c6-b9ad-bf3109c982bc/volumes" Jul 23 17:33:47.964825 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:47.963969 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:55940->10.132.0.43:15021: read: connection reset by peer" start-of-body= Jul 23 17:33:47.964825 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:47.964049 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:55940->10.132.0.43:15021: read: connection reset by peer" Jul 23 17:33:48.686689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.686656 2584 generic.go:356] "Generic (PLEG): container finished" podID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerID="580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e" exitCode=0 Jul 23 17:33:48.686945 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.686726 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerDied","Data":"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e"} Jul 23 17:33:48.687122 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.687108 2584 scope.go:117] "RemoveContainer" containerID="580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e" Jul 23 17:33:48.688067 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.688041 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:48.688121 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.688106 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:48.688159 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:48.688139 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:33:49.693420 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:49.693386 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerStarted","Data":"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6"} Jul 23 17:33:50.161569 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:50.161483 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:50.161726 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:50.161692 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 17:33:50.161770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:50.161740 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 17:33:51.161299 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:51.161268 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 17:33:51.161701 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:51.161330 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 17:33:52.161717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:52.161679 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 17:33:52.162162 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:52.161739 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 17:33:53.161175 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:53.161136 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 17:33:53.161347 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:53.161197 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 17:33:54.161789 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:54.161753 2584 patch_prober.go:28] interesting pod/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 17:33:54.162163 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:54.161808 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 17:33:55.164096 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:55.164071 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:55.164537 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:55.164315 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:33:55.165090 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:33:55.165071 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:34:13.414039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:13.414002 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:34:13.414476 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:13.414220 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" containerID="cri-o://bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6" gracePeriod=30 Jul 23 17:34:14.321950 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.321909 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:34:14.322302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322290 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.322302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322303 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.322418 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322312 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.322418 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322318 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.322418 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322413 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.322509 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.322424 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ffe670-6e31-46c6-b9ad-bf3109c982bc" containerName="istio-proxy" Jul 23 17:34:14.325573 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.325551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.328797 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.328770 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0e2114d4-openshift-default-dockercfg-t5k95\"" Jul 23 17:34:14.338285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.338262 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:34:14.398916 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.398885 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399091 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.398923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399091 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.398947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmc4h\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399091 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399003 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399091 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399021 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399091 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399040 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399327 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399091 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399327 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.399327 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.399240 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500288 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500257 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500316 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500352 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500402 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500549 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmc4h\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.500770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500690 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500792 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500903 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500906 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.500972 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.501066 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.501592 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.501569 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.502569 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.502544 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.502796 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.502779 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.509443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.509406 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.509530 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.509479 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmc4h\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h\") pod \"e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.637946 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.637840 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:14.788163 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.788139 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:34:14.790237 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:34:14.790205 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a275fa_d1bc_4652_8f57_2e6fff6918dd.slice/crio-6c4eeb50213297d637bb749169115a55a42feb4259f4dea0961e5600b71da80e WatchSource:0}: Error finding container 6c4eeb50213297d637bb749169115a55a42feb4259f4dea0961e5600b71da80e: Status 404 returned error can't find the container with id 6c4eeb50213297d637bb749169115a55a42feb4259f4dea0961e5600b71da80e Jul 23 17:34:14.792521 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.792488 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:14.792631 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.792568 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:14.792631 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.792614 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:14.827773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:14.827745 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerStarted","Data":"6c4eeb50213297d637bb749169115a55a42feb4259f4dea0961e5600b71da80e"} Jul 23 17:34:15.835520 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:15.835483 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerStarted","Data":"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009"} Jul 23 17:34:15.863806 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:15.863749 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podStartSLOduration=1.863732613 podStartE2EDuration="1.863732613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:34:15.862525754 +0000 UTC m=+1777.216387156" watchObservedRunningTime="2026-07-23 17:34:15.863732613 +0000 UTC m=+1777.217593992" Jul 23 17:34:16.638224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:16.638184 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:16.643089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:16.643065 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:16.840708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:16.840668 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:16.841567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:16.841548 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:18.679866 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.679809 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:34:18.745124 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745093 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745124 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745129 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745162 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745208 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745233 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd8mf\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745258 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745299 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745377 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745332 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745421 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:34:18.745676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745442 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket\") pod \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\" (UID: \"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6\") " Jul 23 17:34:18.745676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745637 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:34:18.745823 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745641 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:34:18.745823 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745789 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:34:18.746018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.745986 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.746018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.746011 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.746208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.746025 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.746208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.746039 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.746208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.746157 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:34:18.747620 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.747600 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:34:18.747748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.747730 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf" (OuterVolumeSpecName: "kube-api-access-rd8mf") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "kube-api-access-rd8mf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:34:18.747893 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.747869 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:34:18.747893 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.747874 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" (UID: "ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:34:18.846638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.846607 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.846638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.846632 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.846638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.846643 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd8mf\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-kube-api-access-rd8mf\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.846891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.846654 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.846891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.846663 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:18.852468 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.852439 2584 generic.go:356] "Generic (PLEG): container finished" podID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerID="bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6" exitCode=0 Jul 23 17:34:18.852612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.852472 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerDied","Data":"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6"} Jul 23 17:34:18.852612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.852514 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" Jul 23 17:34:18.852612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.852536 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx" event={"ID":"ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6","Type":"ContainerDied","Data":"e3bb444a0570c4e3d97692c7647afc4a9789a1413c6ec3030a073bc767633fbd"} Jul 23 17:34:18.852612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.852559 2584 scope.go:117] "RemoveContainer" containerID="bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6" Jul 23 17:34:18.862285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.862267 2584 scope.go:117] "RemoveContainer" containerID="580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e" Jul 23 17:34:18.870573 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.870554 2584 scope.go:117] "RemoveContainer" containerID="bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6" Jul 23 17:34:18.870860 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:34:18.870840 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6\": container with ID starting with bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6 not found: ID does not exist" containerID="bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6" Jul 23 17:34:18.870909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.870870 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6"} err="failed to get container status \"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6\": rpc error: code = NotFound desc = could not find container \"bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6\": container with ID starting with bbb8a2da6a7fda010af87d72e736d7a5e34c26a0c13aee4ec7ffb87458c3f4c6 not found: ID does not exist" Jul 23 17:34:18.870909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.870889 2584 scope.go:117] "RemoveContainer" containerID="580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e" Jul 23 17:34:18.871143 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:34:18.871127 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e\": container with ID starting with 580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e not found: ID does not exist" containerID="580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e" Jul 23 17:34:18.871182 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.871148 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e"} err="failed to get container status \"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e\": rpc error: code = NotFound desc = could not find container \"580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e\": container with ID starting with 580231b0fd9a4c1d2c40a7d805ec216488800748b4f9bac207be05144154279e not found: ID does not exist" Jul 23 17:34:18.877676 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.877650 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:34:18.884791 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:18.884767 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ab1734b0-gw-openshift-default-666d78cdbc-8wtrx"] Jul 23 17:34:19.131444 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:19.131416 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" path="/var/lib/kubelet/pods/ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6/volumes" Jul 23 17:34:30.916654 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.916560 2584 generic.go:356] "Generic (PLEG): container finished" podID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerID="33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009" exitCode=0 Jul 23 17:34:30.917097 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.916654 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerDied","Data":"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009"} Jul 23 17:34:30.917097 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.917089 2584 scope.go:117] "RemoveContainer" containerID="33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009" Jul 23 17:34:30.918210 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.918178 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:30.918315 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.918251 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:30.918315 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:30.918284 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:34:31.841230 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:31.841195 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:31.923476 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:31.923448 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerStarted","Data":"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe"} Jul 23 17:34:32.638945 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:32.638914 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:32.640043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:32.640021 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:32.640149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:32.640063 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:33.638758 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:33.638729 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:33.639122 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:33.638774 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:34.639033 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:34.638988 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:34.639567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:34.639058 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:35.639069 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:35.639030 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:35.639553 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:35.639088 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:36.638704 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:36.638665 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:36.638934 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:36.638744 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:37.638810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:37.638770 2584 patch_prober.go:28] interesting pod/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 17:34:37.639241 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:37.638825 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 17:34:38.642499 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:38.642469 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:38.642958 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:38.642805 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:38.643479 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:38.643458 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:34:39.182289 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:39.182263 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:34:39.182567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:39.182404 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:34:43.109445 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109409 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn"] Jul 23 17:34:43.109844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109796 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:34:43.109844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109808 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:34:43.109844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109815 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:34:43.109844 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109820 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:34:43.110018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.109874 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:34:43.118003 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.117973 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:43.120513 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.120482 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn"] Jul 23 17:34:43.120675 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.120661 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:34:43.120786 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.120754 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 17:34:43.175907 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.175863 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn\" (UID: \"c984e500-e9b7-49a1-83be-f5817b525139\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:43.276757 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.276722 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn\" (UID: \"c984e500-e9b7-49a1-83be-f5817b525139\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:43.277347 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.277329 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn\" (UID: \"c984e500-e9b7-49a1-83be-f5817b525139\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:43.430870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.430778 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:43.586818 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.586793 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn"] Jul 23 17:34:43.588719 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:34:43.588669 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc984e500_e9b7_49a1_83be_f5817b525139.slice/crio-9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f WatchSource:0}: Error finding container 9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f: Status 404 returned error can't find the container with id 9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f Jul 23 17:34:43.984179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.984139 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" event={"ID":"c984e500-e9b7-49a1-83be-f5817b525139","Type":"ContainerStarted","Data":"4f4071a3f0fba46b82374bca38c56908d15c0e4397f9155e146256b363c8bcfc"} Jul 23 17:34:43.984417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:43.984202 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" event={"ID":"c984e500-e9b7-49a1-83be-f5817b525139","Type":"ContainerStarted","Data":"9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f"} Jul 23 17:34:44.002124 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:44.002069 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" podStartSLOduration=1.002051302 podStartE2EDuration="1.002051302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:34:43.999294308 +0000 UTC m=+1805.353155689" watchObservedRunningTime="2026-07-23 17:34:44.002051302 +0000 UTC m=+1805.355912681" Jul 23 17:34:44.990385 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:44.990327 2584 generic.go:356] "Generic (PLEG): container finished" podID="c984e500-e9b7-49a1-83be-f5817b525139" containerID="4f4071a3f0fba46b82374bca38c56908d15c0e4397f9155e146256b363c8bcfc" exitCode=0 Jul 23 17:34:44.990789 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:44.990414 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" event={"ID":"c984e500-e9b7-49a1-83be-f5817b525139","Type":"ContainerDied","Data":"4f4071a3f0fba46b82374bca38c56908d15c0e4397f9155e146256b363c8bcfc"} Jul 23 17:34:46.133679 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:46.133655 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:46.205774 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:46.205739 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca\") pod \"c984e500-e9b7-49a1-83be-f5817b525139\" (UID: \"c984e500-e9b7-49a1-83be-f5817b525139\") " Jul 23 17:34:46.206090 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:46.206066 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c984e500-e9b7-49a1-83be-f5817b525139" (UID: "c984e500-e9b7-49a1-83be-f5817b525139"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:34:46.307601 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:46.307513 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c984e500-e9b7-49a1-83be-f5817b525139-maas-api-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:34:47.001730 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:47.001708 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" Jul 23 17:34:47.001860 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:47.001731 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn" event={"ID":"c984e500-e9b7-49a1-83be-f5817b525139","Type":"ContainerDied","Data":"9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f"} Jul 23 17:34:47.001860 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:47.001789 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bdf83af729aabddb22fe06ec5d68530c9f7f47cc507d13e957a6dba2622435f" Jul 23 17:34:58.201537 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:58.201502 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn"] Jul 23 17:34:58.203388 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:58.203342 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e2114d4-cvjcn"] Jul 23 17:34:59.131285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:59.131251 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c984e500-e9b7-49a1-83be-f5817b525139" path="/var/lib/kubelet/pods/c984e500-e9b7-49a1-83be-f5817b525139/volumes" Jul 23 17:34:59.502666 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:59.502628 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:34:59.503174 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:34:59.502868 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" containerID="cri-o://3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe" gracePeriod=30 Jul 23 17:35:00.308538 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.308506 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:00.308928 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.308914 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c984e500-e9b7-49a1-83be-f5817b525139" containerName="revoke-keys" Jul 23 17:35:00.308976 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.308931 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c984e500-e9b7-49a1-83be-f5817b525139" containerName="revoke-keys" Jul 23 17:35:00.309022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.309012 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c984e500-e9b7-49a1-83be-f5817b525139" containerName="revoke-keys" Jul 23 17:35:00.309061 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.309027 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab39f0c1-fb36-4ce6-ad0b-11b4a20749e6" containerName="istio-proxy" Jul 23 17:35:00.313331 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.313311 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.316476 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.316453 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e6ac8a29-openshift-default-dockercfg-77r79\"" Jul 23 17:35:00.334223 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.334195 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:00.439151 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439113 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439231 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dw7\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439285 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439304 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439324 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439382 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439450 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.439671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.439486 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.540773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540729 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540786 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dw7\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540880 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540913 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540954 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.540980 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.541282 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.541428 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.541452 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.541692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.541510 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.542076 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.542054 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.543335 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.543313 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.543917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.543897 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.552960 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.552937 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.553316 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.553292 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dw7\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7\") pod \"e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.626788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.626694 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:00.790556 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.790528 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:00.792977 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:35:00.792939 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b312986_feb2_43c7_b749_acfe8f797f3e.slice/crio-b12a843e6cf901ab0ea888a2e5721f7b60e485b8fdbeb6adb92cccfcd141b759 WatchSource:0}: Error finding container b12a843e6cf901ab0ea888a2e5721f7b60e485b8fdbeb6adb92cccfcd141b759: Status 404 returned error can't find the container with id b12a843e6cf901ab0ea888a2e5721f7b60e485b8fdbeb6adb92cccfcd141b759 Jul 23 17:35:00.795831 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.795791 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:00.795941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.795877 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:00.795941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:00.795922 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:01.074521 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:01.074477 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerStarted","Data":"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7"} Jul 23 17:35:01.074737 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:01.074534 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerStarted","Data":"b12a843e6cf901ab0ea888a2e5721f7b60e485b8fdbeb6adb92cccfcd141b759"} Jul 23 17:35:01.627194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:01.627158 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:01.631948 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:01.631918 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:01.657327 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:01.657277 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podStartSLOduration=1.657261104 podStartE2EDuration="1.657261104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:35:01.110572381 +0000 UTC m=+1822.464433762" watchObservedRunningTime="2026-07-23 17:35:01.657261104 +0000 UTC m=+1823.011122483" Jul 23 17:35:02.080161 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:02.080131 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:02.081170 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:02.081153 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:04.767766 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.767699 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:35:04.886662 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886610 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.886858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886689 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.886858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886711 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.886858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886734 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.886858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886796 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmc4h\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.886858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886824 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.887117 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886871 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.887117 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886917 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.887117 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.886943 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token\") pod \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\" (UID: \"41a275fa-d1bc-4652-8f57-2e6fff6918dd\") " Jul 23 17:35:04.887117 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887036 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:04.887318 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887201 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:04.887318 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887295 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:04.887452 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887419 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:35:04.887507 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887448 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.887507 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887468 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.887507 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887484 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.887853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.887831 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:04.889104 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.889087 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:04.889207 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.889186 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h" (OuterVolumeSpecName: "kube-api-access-gmc4h") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "kube-api-access-gmc4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:35:04.889404 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.889387 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:35:04.889460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.889427 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41a275fa-d1bc-4652-8f57-2e6fff6918dd" (UID: "41a275fa-d1bc-4652-8f57-2e6fff6918dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:35:04.988237 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988185 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.988237 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988228 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.988237 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988237 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmc4h\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-kube-api-access-gmc4h\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.988237 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988246 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.988237 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988257 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:04.988590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:04.988265 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a275fa-d1bc-4652-8f57-2e6fff6918dd-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:05.096891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.096805 2584 generic.go:356] "Generic (PLEG): container finished" podID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerID="3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe" exitCode=0 Jul 23 17:35:05.096891 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.096850 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" Jul 23 17:35:05.097100 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.096892 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerDied","Data":"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe"} Jul 23 17:35:05.097100 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.096946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x" event={"ID":"41a275fa-d1bc-4652-8f57-2e6fff6918dd","Type":"ContainerDied","Data":"6c4eeb50213297d637bb749169115a55a42feb4259f4dea0961e5600b71da80e"} Jul 23 17:35:05.097100 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.096971 2584 scope.go:117] "RemoveContainer" containerID="3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe" Jul 23 17:35:05.106785 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.106768 2584 scope.go:117] "RemoveContainer" containerID="33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009" Jul 23 17:35:05.115271 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.115251 2584 scope.go:117] "RemoveContainer" containerID="3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe" Jul 23 17:35:05.115538 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:35:05.115518 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe\": container with ID starting with 3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe not found: ID does not exist" containerID="3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe" Jul 23 17:35:05.115599 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.115559 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe"} err="failed to get container status \"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe\": rpc error: code = NotFound desc = could not find container \"3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe\": container with ID starting with 3e53d2762e5f3e0494902f7ce7cc82bcefb07ff329b9a5c4ae63a4686388bafe not found: ID does not exist" Jul 23 17:35:05.115599 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.115578 2584 scope.go:117] "RemoveContainer" containerID="33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009" Jul 23 17:35:05.115823 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:35:05.115808 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009\": container with ID starting with 33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009 not found: ID does not exist" containerID="33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009" Jul 23 17:35:05.115865 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.115827 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009"} err="failed to get container status \"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009\": rpc error: code = NotFound desc = could not find container \"33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009\": container with ID starting with 33d2d0045070d8ff4bc64e57bba17dba44f971307afa717bd41fe3cc428bd009 not found: ID does not exist" Jul 23 17:35:05.121739 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.121717 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:35:05.131597 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:05.131575 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0e2114d4-openshift-default-6d7bb48b85-4w88x"] Jul 23 17:35:07.131249 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:07.131213 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" path="/var/lib/kubelet/pods/41a275fa-d1bc-4652-8f57-2e6fff6918dd/volumes" Jul 23 17:35:17.326081 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:17.324770 2584 patch_prober.go:28] interesting pod/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:49070->10.132.0.46:15021: read: connection reset by peer" start-of-body= Jul 23 17:35:17.326081 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:17.324845 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:49070->10.132.0.46:15021: read: connection reset by peer" Jul 23 17:35:18.170421 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.170390 2584 generic.go:356] "Generic (PLEG): container finished" podID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerID="6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7" exitCode=0 Jul 23 17:35:18.170600 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.170456 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerDied","Data":"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7"} Jul 23 17:35:18.170853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.170840 2584 scope.go:117] "RemoveContainer" containerID="6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7" Jul 23 17:35:18.171902 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.171873 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:18.171958 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.171944 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:18.171994 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:18.171975 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:35:19.177025 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:19.176993 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerStarted","Data":"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c"} Jul 23 17:35:19.627096 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:19.627058 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:19.627352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:19.627329 2584 patch_prober.go:28] interesting pod/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 23 17:35:19.627431 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:19.627406 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 23 17:35:20.628079 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:20.628041 2584 patch_prober.go:28] interesting pod/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 23 17:35:20.628534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:20.628099 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 23 17:35:21.627422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:21.627381 2584 patch_prober.go:28] interesting pod/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 23 17:35:21.627611 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:21.627444 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 23 17:35:22.627739 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:22.627702 2584 patch_prober.go:28] interesting pod/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 23 17:35:22.628150 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:22.627763 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 23 17:35:23.631186 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:23.631155 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:23.631608 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:23.631395 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:23.632070 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:23.632057 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:42.651255 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:42.651220 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:42.651800 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:42.651462 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" containerID="cri-o://d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c" gracePeriod=30 Jul 23 17:35:47.919118 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.919048 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:47.993302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993270 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993304 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94dw7\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993330 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993410 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993476 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993491 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993525 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993569 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket\") pod \"3b312986-feb2-43c7-b749-acfe8f797f3e\" (UID: \"3b312986-feb2-43c7-b749-acfe8f797f3e\") " Jul 23 17:35:47.993944 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993844 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:35:47.994030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.994006 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:47.994098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.994036 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:47.994098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.993967 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:47.994541 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.994513 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:47.995728 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.995703 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7" (OuterVolumeSpecName: "kube-api-access-94dw7") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "kube-api-access-94dw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:35:47.995834 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.995736 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:35:47.995834 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.995747 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:35:47.995834 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:47.995784 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b312986-feb2-43c7-b749-acfe8f797f3e" (UID: "3b312986-feb2-43c7-b749-acfe8f797f3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:35:48.094982 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.094953 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.094982 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.094979 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.094982 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.094989 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94dw7\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-kube-api-access-94dw7\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.094998 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.095007 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b312986-feb2-43c7-b749-acfe8f797f3e-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.095015 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.095023 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.095030 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b312986-feb2-43c7-b749-acfe8f797f3e-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.095198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.095039 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b312986-feb2-43c7-b749-acfe8f797f3e-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:35:48.328817 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.328780 2584 generic.go:356] "Generic (PLEG): container finished" podID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerID="d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c" exitCode=0 Jul 23 17:35:48.329021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.328830 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" Jul 23 17:35:48.329021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.328870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerDied","Data":"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c"} Jul 23 17:35:48.329021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.328925 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j" event={"ID":"3b312986-feb2-43c7-b749-acfe8f797f3e","Type":"ContainerDied","Data":"b12a843e6cf901ab0ea888a2e5721f7b60e485b8fdbeb6adb92cccfcd141b759"} Jul 23 17:35:48.329021 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.328949 2584 scope.go:117] "RemoveContainer" containerID="d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c" Jul 23 17:35:48.340034 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.340015 2584 scope.go:117] "RemoveContainer" containerID="6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7" Jul 23 17:35:48.347980 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.347953 2584 scope.go:117] "RemoveContainer" containerID="d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c" Jul 23 17:35:48.348286 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:35:48.348258 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c\": container with ID starting with d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c not found: ID does not exist" containerID="d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c" Jul 23 17:35:48.348567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.348302 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c"} err="failed to get container status \"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c\": rpc error: code = NotFound desc = could not find container \"d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c\": container with ID starting with d7e8232457a52a20c34c2931c589f107fc6980b5fa876eb58e5198b36a8b313c not found: ID does not exist" Jul 23 17:35:48.348567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.348325 2584 scope.go:117] "RemoveContainer" containerID="6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7" Jul 23 17:35:48.348709 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:35:48.348683 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7\": container with ID starting with 6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7 not found: ID does not exist" containerID="6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7" Jul 23 17:35:48.348747 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.348719 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7"} err="failed to get container status \"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7\": rpc error: code = NotFound desc = could not find container \"6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7\": container with ID starting with 6a063a89ca95a537d07e9de3fcec4c1c2819e21f2f59536461eab629ec0ec8e7 not found: ID does not exist" Jul 23 17:35:48.350111 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.350091 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:48.361896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:48.361874 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e6ac8a29-openshift-default-5b74d7c9ff-wz78j"] Jul 23 17:35:49.131917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:35:49.131882 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" path="/var/lib/kubelet/pods/3b312986-feb2-43c7-b749-acfe8f797f3e/volumes" Jul 23 17:37:25.345424 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345389 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:37:25.345877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345828 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:37:25.345877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345842 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:37:25.345877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345858 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:37:25.345877 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345865 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:37:25.346015 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345932 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:37:25.346015 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345941 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:37:25.346015 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345947 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:37:25.346015 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.345953 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:37:25.348987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.348970 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.352870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.352845 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c99bbc71-openshift-default-dockercfg-s2zzt\"" Jul 23 17:37:25.377118 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.377092 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:37:25.445105 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445136 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445167 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q9jx\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445226 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445594 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445324 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445594 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445388 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.445594 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.445419 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546692 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546737 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546757 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q9jx\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546779 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546807 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546889 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.546905 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.546910 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.547329 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.547232 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.547329 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.547265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.547464 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.547384 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.547464 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.547436 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.547713 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.547692 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.549240 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.549219 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.549539 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.549521 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.555339 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.555316 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.555459 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.555351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q9jx\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx\") pod \"e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.659496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.659403 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:25.803783 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.803758 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:37:25.805586 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:37:25.805561 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7041775_d9b8_4f0a_87a2_da38571a8b27.slice/crio-44f1ae9debe4c68f1e577cb5fcc9313cedc33b38d2a70685b304ce0285921ad5 WatchSource:0}: Error finding container 44f1ae9debe4c68f1e577cb5fcc9313cedc33b38d2a70685b304ce0285921ad5: Status 404 returned error can't find the container with id 44f1ae9debe4c68f1e577cb5fcc9313cedc33b38d2a70685b304ce0285921ad5 Jul 23 17:37:25.807719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.807684 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:25.807801 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.807747 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:25.807801 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.807777 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:25.819567 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:25.819544 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerStarted","Data":"44f1ae9debe4c68f1e577cb5fcc9313cedc33b38d2a70685b304ce0285921ad5"} Jul 23 17:37:26.828878 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:26.828847 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerStarted","Data":"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472"} Jul 23 17:37:26.853533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:26.853482 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podStartSLOduration=1.853467865 podStartE2EDuration="1.853467865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:37:26.851057511 +0000 UTC m=+1968.204918893" watchObservedRunningTime="2026-07-23 17:37:26.853467865 +0000 UTC m=+1968.207329245" Jul 23 17:37:27.660480 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:27.660437 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:27.665694 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:27.665664 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:27.833798 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:27.833769 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:27.834804 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:27.834786 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:41.903871 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.903838 2584 generic.go:356] "Generic (PLEG): container finished" podID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerID="c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472" exitCode=0 Jul 23 17:37:41.904306 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.903912 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerDied","Data":"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472"} Jul 23 17:37:41.904306 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.904291 2584 scope.go:117] "RemoveContainer" containerID="c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472" Jul 23 17:37:41.905314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.905290 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:41.905390 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.905353 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:41.905432 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:41.905405 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:37:42.834345 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:42.834310 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:42.915273 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:42.915238 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerStarted","Data":"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96"} Jul 23 17:37:43.659807 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:43.659774 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:43.660938 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:43.660911 2584 patch_prober.go:28] interesting pod/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 23 17:37:43.661051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:43.660961 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 23 17:37:44.660386 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:44.660326 2584 patch_prober.go:28] interesting pod/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 23 17:37:44.660815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:44.660440 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 23 17:37:45.660376 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:45.660330 2584 patch_prober.go:28] interesting pod/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 23 17:37:45.660777 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:45.660420 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 23 17:37:46.660152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:46.660108 2584 patch_prober.go:28] interesting pod/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 23 17:37:46.660381 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:46.660180 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 23 17:37:47.664119 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:47.664090 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:47.664534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:47.664378 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:37:47.665083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:37:47.665065 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:38:04.680004 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:04.679964 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:38:04.680498 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:04.680219 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" containerID="cri-o://4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96" gracePeriod=30 Jul 23 17:38:09.928248 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.928221 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:38:09.960935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.960867 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.960935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.960899 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.960937 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.960960 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.960989 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961037 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961067 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q9jx\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961136 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:38:09.961497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961221 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961229 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:38:09.961497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961287 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs\") pod \"b7041775-d9b8-4f0a-87a2-da38571a8b27\" (UID: \"b7041775-d9b8-4f0a-87a2-da38571a8b27\") " Jul 23 17:38:09.961726 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961680 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:09.961726 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961701 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:09.961953 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.961920 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:38:09.962434 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.962207 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:38:09.962434 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.962391 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:38:09.964118 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.964093 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:38:09.964232 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.964160 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:38:09.964863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.964831 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:38:09.964863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:09.964846 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx" (OuterVolumeSpecName: "kube-api-access-4q9jx") pod "b7041775-d9b8-4f0a-87a2-da38571a8b27" (UID: "b7041775-d9b8-4f0a-87a2-da38571a8b27"). InnerVolumeSpecName "kube-api-access-4q9jx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:38:10.060523 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.060488 2584 generic.go:356] "Generic (PLEG): container finished" podID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerID="4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96" exitCode=0 Jul 23 17:38:10.060691 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.060539 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" Jul 23 17:38:10.060691 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.060541 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerDied","Data":"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96"} Jul 23 17:38:10.060691 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.060595 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp" event={"ID":"b7041775-d9b8-4f0a-87a2-da38571a8b27","Type":"ContainerDied","Data":"44f1ae9debe4c68f1e577cb5fcc9313cedc33b38d2a70685b304ce0285921ad5"} Jul 23 17:38:10.060691 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.060626 2584 scope.go:117] "RemoveContainer" containerID="4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96" Jul 23 17:38:10.062308 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062286 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q9jx\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-kube-api-access-4q9jx\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062308 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062308 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062317 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062325 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062332 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062340 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7041775-d9b8-4f0a-87a2-da38571a8b27-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.062590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.062349 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7041775-d9b8-4f0a-87a2-da38571a8b27-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:38:10.069996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.069985 2584 scope.go:117] "RemoveContainer" containerID="c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472" Jul 23 17:38:10.077803 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.077785 2584 scope.go:117] "RemoveContainer" containerID="4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96" Jul 23 17:38:10.078050 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:38:10.078030 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96\": container with ID starting with 4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96 not found: ID does not exist" containerID="4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96" Jul 23 17:38:10.078099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.078058 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96"} err="failed to get container status \"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96\": rpc error: code = NotFound desc = could not find container \"4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96\": container with ID starting with 4c31732240ac6c6eb768bdb46a1964ce64a912a9001df743be77fc9fada3df96 not found: ID does not exist" Jul 23 17:38:10.078099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.078075 2584 scope.go:117] "RemoveContainer" containerID="c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472" Jul 23 17:38:10.078321 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:38:10.078304 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472\": container with ID starting with c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472 not found: ID does not exist" containerID="c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472" Jul 23 17:38:10.078415 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.078323 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472"} err="failed to get container status \"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472\": rpc error: code = NotFound desc = could not find container \"c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472\": container with ID starting with c5526a49b716ce6b2b8502e140089e2a091d18bc574598b39fcf39d41506f472 not found: ID does not exist" Jul 23 17:38:10.098217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.098193 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:38:10.104760 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:10.104737 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c99bbc71-openshift-default-7dc8b54c75-x2ljp"] Jul 23 17:38:11.130868 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:11.130835 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" path="/var/lib/kubelet/pods/b7041775-d9b8-4f0a-87a2-da38571a8b27/volumes" Jul 23 17:38:30.593784 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.593752 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594130 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594140 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b312986-feb2-43c7-b749-acfe8f797f3e" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594150 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594155 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594169 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594175 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a275fa-d1bc-4652-8f57-2e6fff6918dd" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594250 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:38:30.594392 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.594261 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:38:30.597406 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.597387 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.600586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.600563 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7a6850d7-openshift-default-dockercfg-q55qn\"" Jul 23 17:38:30.622301 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.622275 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:38:30.751769 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751731 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xkw7\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.751944 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751779 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.751944 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751846 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751978 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.751996 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.752011 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.752037 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.752052 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.752052 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.852863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.852863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852815 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.852863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852834 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.852863 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852855 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852894 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852917 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.852983 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xkw7\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853046 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853255 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853278 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853645 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853611 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853815 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853792 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.853914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.853861 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.855455 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.855430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.855674 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.855647 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.864802 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.864783 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.865061 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.865044 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xkw7\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7\") pod \"e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:30.910010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:30.909984 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:31.077520 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:38:31.077489 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0f9f2f_e739_4168_8b46_3363d868e4f6.slice/crio-7421cdd4eb20f47188665840a451d6938a557fd05f2812ffafa23451aad2af47 WatchSource:0}: Error finding container 7421cdd4eb20f47188665840a451d6938a557fd05f2812ffafa23451aad2af47: Status 404 returned error can't find the container with id 7421cdd4eb20f47188665840a451d6938a557fd05f2812ffafa23451aad2af47 Jul 23 17:38:31.079995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.079959 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:31.080121 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.080029 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:31.080121 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.080059 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:31.081716 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.081696 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:38:31.161830 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.161804 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerStarted","Data":"8cab42b43d1fe85cdf48b0e52720dbb8db7adfc563eab1dcd77466c4c85e6740"} Jul 23 17:38:31.161945 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.161850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerStarted","Data":"7421cdd4eb20f47188665840a451d6938a557fd05f2812ffafa23451aad2af47"} Jul 23 17:38:31.211457 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.211392 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podStartSLOduration=1.211353427 podStartE2EDuration="1.211353427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:38:31.209141064 +0000 UTC m=+2032.563002448" watchObservedRunningTime="2026-07-23 17:38:31.211353427 +0000 UTC m=+2032.565214811" Jul 23 17:38:31.910259 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:31.910227 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:32.915153 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:32.915121 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:33.172133 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:33.172052 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:33.172963 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:33.172942 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:48.173949 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.173913 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 23 17:38:48.174347 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.173979 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 23 17:38:48.252822 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.252792 2584 generic.go:356] "Generic (PLEG): container finished" podID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerID="8cab42b43d1fe85cdf48b0e52720dbb8db7adfc563eab1dcd77466c4c85e6740" exitCode=0 Jul 23 17:38:48.252969 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.252863 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerDied","Data":"8cab42b43d1fe85cdf48b0e52720dbb8db7adfc563eab1dcd77466c4c85e6740"} Jul 23 17:38:48.253195 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.253178 2584 scope.go:117] "RemoveContainer" containerID="8cab42b43d1fe85cdf48b0e52720dbb8db7adfc563eab1dcd77466c4c85e6740" Jul 23 17:38:48.254189 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.254161 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:48.254250 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.254226 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:48.254285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:48.254257 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:38:49.259871 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:49.259836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerStarted","Data":"e78f168b44839682ebd9e0232fe26cd6e0002baf50385786b8d2eabaa3e5caf3"} Jul 23 17:38:49.911346 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:49.911311 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:49.911568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:49.911446 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 23 17:38:49.911568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:49.911488 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 23 17:38:50.910613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:50.910576 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 23 17:38:50.911223 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:50.911191 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 23 17:38:51.911166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:51.911127 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 23 17:38:51.911776 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:51.911741 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 23 17:38:52.910708 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:52.910673 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 23 17:38:52.910982 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:52.910953 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 23 17:38:53.915158 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:53.915131 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:53.915542 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:53.915478 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:38:53.916205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:38:53.916189 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:39:11.246597 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.246563 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr"] Jul 23 17:39:11.247219 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.246993 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:39:11.247219 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.247009 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7041775-d9b8-4f0a-87a2-da38571a8b27" containerName="istio-proxy" Jul 23 17:39:11.250423 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.250398 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:11.253572 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.253340 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 17:39:11.253572 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.253466 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:39:11.257771 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.257745 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr"] Jul 23 17:39:11.307128 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.307103 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr\" (UID: \"44ab57be-7d75-478a-b5e4-69465b366205\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:11.408428 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.408402 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr\" (UID: \"44ab57be-7d75-478a-b5e4-69465b366205\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:11.408987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.408971 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr\" (UID: \"44ab57be-7d75-478a-b5e4-69465b366205\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:11.563047 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.562987 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:11.703964 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:11.703941 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr"] Jul 23 17:39:11.705711 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:39:11.705684 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ab57be_7d75_478a_b5e4_69465b366205.slice/crio-840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113 WatchSource:0}: Error finding container 840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113: Status 404 returned error can't find the container with id 840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113 Jul 23 17:39:12.377608 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:12.377577 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" event={"ID":"44ab57be-7d75-478a-b5e4-69465b366205","Type":"ContainerStarted","Data":"0043ca2014b5042d3ee7686d8e84258fd9d02168a2039a0d461e18c2ee499f0f"} Jul 23 17:39:12.378040 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:12.377630 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" event={"ID":"44ab57be-7d75-478a-b5e4-69465b366205","Type":"ContainerStarted","Data":"840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113"} Jul 23 17:39:12.401944 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:12.401899 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" podStartSLOduration=1.401884514 podStartE2EDuration="1.401884514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:39:12.400925064 +0000 UTC m=+2073.754786470" watchObservedRunningTime="2026-07-23 17:39:12.401884514 +0000 UTC m=+2073.755745894" Jul 23 17:39:13.391419 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:13.391347 2584 generic.go:356] "Generic (PLEG): container finished" podID="44ab57be-7d75-478a-b5e4-69465b366205" containerID="0043ca2014b5042d3ee7686d8e84258fd9d02168a2039a0d461e18c2ee499f0f" exitCode=0 Jul 23 17:39:13.391419 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:13.391395 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" event={"ID":"44ab57be-7d75-478a-b5e4-69465b366205","Type":"ContainerDied","Data":"0043ca2014b5042d3ee7686d8e84258fd9d02168a2039a0d461e18c2ee499f0f"} Jul 23 17:39:14.532143 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:14.532121 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:14.632963 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:14.632939 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca\") pod \"44ab57be-7d75-478a-b5e4-69465b366205\" (UID: \"44ab57be-7d75-478a-b5e4-69465b366205\") " Jul 23 17:39:14.633295 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:14.633274 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44ab57be-7d75-478a-b5e4-69465b366205" (UID: "44ab57be-7d75-478a-b5e4-69465b366205"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:39:14.734075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:14.734051 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44ab57be-7d75-478a-b5e4-69465b366205-maas-api-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:39:15.403343 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:15.403268 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" event={"ID":"44ab57be-7d75-478a-b5e4-69465b366205","Type":"ContainerDied","Data":"840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113"} Jul 23 17:39:15.403343 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:15.403335 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="840597fd2021dff663db8d975806b66ec1363065681cdee228517e3e3ade1113" Jul 23 17:39:15.403533 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:15.403279 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr" Jul 23 17:39:31.387231 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:31.387200 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr"] Jul 23 17:39:31.390642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:31.390618 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7a6850d7-dfckr"] Jul 23 17:39:33.132018 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:33.131985 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ab57be-7d75-478a-b5e4-69465b366205" path="/var/lib/kubelet/pods/44ab57be-7d75-478a-b5e4-69465b366205/volumes" Jul 23 17:39:39.212137 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:39.212107 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:39:39.214672 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:39.214651 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:39:39.269458 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:39:39.269433 2584 scope.go:117] "RemoveContainer" containerID="92fe1e084fc98ac06daf1c4781f853ed3c86411458c6b7f057da1e7f42586140" Jul 23 17:40:22.583161 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:22.583132 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:40:22.583702 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:22.583327 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" containerID="cri-o://e78f168b44839682ebd9e0232fe26cd6e0002baf50385786b8d2eabaa3e5caf3" gracePeriod=30 Jul 23 17:40:23.917914 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:23.917876 2584 patch_prober.go:28] interesting pod/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 17:40:23.918280 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:23.917934 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 17:40:27.764795 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.764770 2584 generic.go:356] "Generic (PLEG): container finished" podID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerID="e78f168b44839682ebd9e0232fe26cd6e0002baf50385786b8d2eabaa3e5caf3" exitCode=0 Jul 23 17:40:27.765166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.764832 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerDied","Data":"e78f168b44839682ebd9e0232fe26cd6e0002baf50385786b8d2eabaa3e5caf3"} Jul 23 17:40:27.765166 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.764893 2584 scope.go:117] "RemoveContainer" containerID="8cab42b43d1fe85cdf48b0e52720dbb8db7adfc563eab1dcd77466c4c85e6740" Jul 23 17:40:27.837276 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.837257 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:40:27.914615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914560 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xkw7\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914597 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914767 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914657 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914821 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914806 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914875 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914849 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914928 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914881 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.914928 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914905 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.915030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914940 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.915030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.914974 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket\") pod \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\" (UID: \"ae0f9f2f-e739-4168-8b46-3363d868e4f6\") " Jul 23 17:40:27.915482 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.915304 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:40:27.915590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.915493 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:40:27.915590 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.915509 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:40:27.915737 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.915619 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:40:27.915737 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.915680 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:40:27.917010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.916971 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7" (OuterVolumeSpecName: "kube-api-access-4xkw7") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "kube-api-access-4xkw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:40:27.917108 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.917047 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:40:27.917276 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.917254 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:40:27.917542 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:27.917525 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae0f9f2f-e739-4168-8b46-3363d868e4f6" (UID: "ae0f9f2f-e739-4168-8b46-3363d868e4f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:40:28.015596 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015573 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xkw7\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-kube-api-access-4xkw7\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015596 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015597 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015609 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015621 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015632 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015642 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015660 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015673 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0f9f2f-e739-4168-8b46-3363d868e4f6-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.015763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.015702 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0f9f2f-e739-4168-8b46-3363d868e4f6-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:40:28.770842 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.770819 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" Jul 23 17:40:28.771256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.770850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn" event={"ID":"ae0f9f2f-e739-4168-8b46-3363d868e4f6","Type":"ContainerDied","Data":"7421cdd4eb20f47188665840a451d6938a557fd05f2812ffafa23451aad2af47"} Jul 23 17:40:28.771256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.770926 2584 scope.go:117] "RemoveContainer" containerID="e78f168b44839682ebd9e0232fe26cd6e0002baf50385786b8d2eabaa3e5caf3" Jul 23 17:40:28.794407 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.794376 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:40:28.800149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:28.800125 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7a6850d7-openshift-default-65d9f844b7-bpvtn"] Jul 23 17:40:29.130968 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:40:29.130912 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" path="/var/lib/kubelet/pods/ae0f9f2f-e739-4168-8b46-3363d868e4f6/volumes" Jul 23 17:41:21.533108 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533075 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533466 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533479 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533487 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533492 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533506 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ab57be-7d75-478a-b5e4-69465b366205" containerName="revoke-keys" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533511 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ab57be-7d75-478a-b5e4-69465b366205" containerName="revoke-keys" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533564 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:21.533719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.533573 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ab57be-7d75-478a-b5e4-69465b366205" containerName="revoke-keys" Jul 23 17:41:21.536534 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.536517 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.538996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.538979 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-219aa1-openshift-default-dockercfg-4p8cl\"" Jul 23 17:41:21.554274 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.554225 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:41:21.620786 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620744 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.620979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620823 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.620979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.620979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620882 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.620979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.620979 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.620971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw5hx\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.621246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.621025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.621246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.621048 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.621246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.621106 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722012 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.721977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722047 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722075 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722135 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722201 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722166 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw5hx\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722538 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722538 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722538 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722508 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722682 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722586 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722737 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722678 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722807 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.722987 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.722970 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.724555 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.724536 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.724712 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.724691 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.731050 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.731027 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.731287 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.731267 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw5hx\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx\") pod \"e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.851256 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.851198 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:21.992798 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.992774 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:41:21.994727 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:41:21.994698 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1204be6_d7a7_4696_88f6_35decfadc1cc.slice/crio-0770bc16e0b6abe28b020bf682db6888b90264204a31a97a3992f5253b9c7c0c WatchSource:0}: Error finding container 0770bc16e0b6abe28b020bf682db6888b90264204a31a97a3992f5253b9c7c0c: Status 404 returned error can't find the container with id 0770bc16e0b6abe28b020bf682db6888b90264204a31a97a3992f5253b9c7c0c Jul 23 17:41:21.997197 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.997167 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:21.997296 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.997231 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:21.997296 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:21.997259 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:22.021618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:22.021589 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerStarted","Data":"0770bc16e0b6abe28b020bf682db6888b90264204a31a97a3992f5253b9c7c0c"} Jul 23 17:41:23.028673 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:23.028637 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerStarted","Data":"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2"} Jul 23 17:41:23.069106 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:23.069058 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podStartSLOduration=2.069043782 podStartE2EDuration="2.069043782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:41:23.068760524 +0000 UTC m=+2204.422621907" watchObservedRunningTime="2026-07-23 17:41:23.069043782 +0000 UTC m=+2204.422905159" Jul 23 17:41:23.851553 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:23.851525 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:23.856491 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:23.856462 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:24.033440 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:24.033419 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:24.034050 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:24.034034 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:30.204386 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.199322 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:41:30.213816 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.211197 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0f9f2f-e739-4168-8b46-3363d868e4f6" containerName="istio-proxy" Jul 23 17:41:30.220168 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.219776 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.220795 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.219624 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:41:30.224324 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.223532 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7363d9-openshift-default-dockercfg-54jkv\"" Jul 23 17:41:30.290288 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290241 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290629 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290642 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290704 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp2wj\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290890 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290770 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290890 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290795 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.290890 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290852 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.291041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290889 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.291041 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.290932 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.392404 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.392354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.392655 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.392637 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.392796 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.392781 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.392929 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.392915 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.393045 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.393032 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.393788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.392811 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.393788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.393439 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.393788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.393651 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.394015 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.393827 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.394454 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.394432 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.394599 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.394143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.394732 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.394716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.395227 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.395211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.395754 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.395736 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp2wj\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.399660 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.399627 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.407977 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.407945 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.414205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.412055 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp2wj\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.420955 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.420930 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo\") pod \"e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.537656 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.537573 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:30.717903 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.717876 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:41:30.720174 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:41:30.720140 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5905a0a_d597_4210_ac8f_cfbaa68e51d1.slice/crio-dcffa6d8fac0e10312748a23862d01046366ccc240effe9682201e58acfc4db6 WatchSource:0}: Error finding container dcffa6d8fac0e10312748a23862d01046366ccc240effe9682201e58acfc4db6: Status 404 returned error can't find the container with id dcffa6d8fac0e10312748a23862d01046366ccc240effe9682201e58acfc4db6 Jul 23 17:41:30.722747 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.722711 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:30.722833 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.722782 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:30.722833 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:30.722811 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:31.070474 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.070440 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerStarted","Data":"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06"} Jul 23 17:41:31.070663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.070498 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerStarted","Data":"dcffa6d8fac0e10312748a23862d01046366ccc240effe9682201e58acfc4db6"} Jul 23 17:41:31.092498 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.092443 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podStartSLOduration=1.092424959 podStartE2EDuration="1.092424959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:41:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:41:31.091626932 +0000 UTC m=+2212.445488321" watchObservedRunningTime="2026-07-23 17:41:31.092424959 +0000 UTC m=+2212.446286340" Jul 23 17:41:31.538422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.538390 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:31.540031 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.540005 2584 patch_prober.go:28] interesting pod/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 23 17:41:31.540140 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:31.540053 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 23 17:41:32.541611 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:32.541577 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:33.079613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:33.079584 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:33.080487 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:33.080466 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:37.102342 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.102310 2584 generic.go:356] "Generic (PLEG): container finished" podID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerID="4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2" exitCode=0 Jul 23 17:41:37.102793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.102399 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerDied","Data":"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2"} Jul 23 17:41:37.102838 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.102793 2584 scope.go:117] "RemoveContainer" containerID="4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2" Jul 23 17:41:37.103896 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.103862 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:37.104010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.103954 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:37.104010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:37.104003 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:38.112967 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:38.112921 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerStarted","Data":"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45"} Jul 23 17:41:38.851532 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:38.851502 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:38.852684 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:38.852663 2584 patch_prober.go:28] interesting pod/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 23 17:41:38.852740 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:38.852720 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 23 17:41:39.033375 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:39.033342 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:39.300818 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:39.300785 2584 scope.go:117] "RemoveContainer" containerID="4f4071a3f0fba46b82374bca38c56908d15c0e4397f9155e146256b363c8bcfc" Jul 23 17:41:39.851938 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:39.851898 2584 patch_prober.go:28] interesting pod/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 23 17:41:39.852110 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:39.851965 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 23 17:41:40.852403 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:40.852347 2584 patch_prober.go:28] interesting pod/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 23 17:41:40.852790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:40.852417 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 23 17:41:41.852096 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:41.852059 2584 patch_prober.go:28] interesting pod/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 23 17:41:41.852283 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:41.852118 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 23 17:41:42.855510 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:42.855474 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:42.856456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:42.856434 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:41:47.177976 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.177907 2584 generic.go:356] "Generic (PLEG): container finished" podID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerID="aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06" exitCode=0 Jul 23 17:41:47.177976 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.177964 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerDied","Data":"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06"} Jul 23 17:41:47.178403 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.178265 2584 scope.go:117] "RemoveContainer" containerID="aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06" Jul 23 17:41:47.179185 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.179157 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:47.179257 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.179240 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:47.179308 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:47.179291 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:41:48.079883 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:48.079844 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:48.185265 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:48.185235 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerStarted","Data":"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503"} Jul 23 17:41:48.538030 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:48.538000 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:48.539144 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:48.539121 2584 patch_prober.go:28] interesting pod/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 23 17:41:48.539275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:48.539163 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 23 17:41:49.538412 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:49.538336 2584 patch_prober.go:28] interesting pod/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 23 17:41:49.538841 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:49.538447 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 23 17:41:50.538771 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:50.538730 2584 patch_prober.go:28] interesting pod/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 23 17:41:50.539198 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:50.538788 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 23 17:41:51.538393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:51.538320 2584 patch_prober.go:28] interesting pod/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 23 17:41:51.538632 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:51.538409 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 23 17:41:52.544070 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:52.544035 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:52.544583 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:52.544313 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:41:52.545132 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:41:52.545114 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:42:38.840425 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.840381 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b"] Jul 23 17:42:38.844485 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.844454 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:38.849010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.848982 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 17:42:38.849195 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.848990 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:42:38.862584 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.862549 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b"] Jul 23 17:42:38.966422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:38.966373 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b\" (UID: \"6158043c-3c1e-4daa-bb56-8d8a1e5b4770\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:39.067328 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.067285 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b\" (UID: \"6158043c-3c1e-4daa-bb56-8d8a1e5b4770\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:39.070249 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.070218 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 17:42:39.078396 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.078344 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b\" (UID: \"6158043c-3c1e-4daa-bb56-8d8a1e5b4770\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:39.159283 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.159199 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wfmw\"" Jul 23 17:42:39.165779 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.165749 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:39.326727 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.326613 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b"] Jul 23 17:42:39.329475 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:42:39.329432 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6158043c_3c1e_4daa_bb56_8d8a1e5b4770.slice/crio-412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf WatchSource:0}: Error finding container 412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf: Status 404 returned error can't find the container with id 412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf Jul 23 17:42:39.455089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:39.455055 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" event={"ID":"6158043c-3c1e-4daa-bb56-8d8a1e5b4770","Type":"ContainerStarted","Data":"412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf"} Jul 23 17:42:40.461556 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:40.461515 2584 generic.go:356] "Generic (PLEG): container finished" podID="6158043c-3c1e-4daa-bb56-8d8a1e5b4770" containerID="8401131e8579695f664c19b6264e3d2dcee705c4ef26403e92e637036a1339ff" exitCode=0 Jul 23 17:42:40.462012 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:40.461597 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" event={"ID":"6158043c-3c1e-4daa-bb56-8d8a1e5b4770","Type":"ContainerDied","Data":"8401131e8579695f664c19b6264e3d2dcee705c4ef26403e92e637036a1339ff"} Jul 23 17:42:41.614664 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:41.614622 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:41.691671 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:41.691620 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca\") pod \"6158043c-3c1e-4daa-bb56-8d8a1e5b4770\" (UID: \"6158043c-3c1e-4daa-bb56-8d8a1e5b4770\") " Jul 23 17:42:41.692013 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:41.691987 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6158043c-3c1e-4daa-bb56-8d8a1e5b4770" (UID: "6158043c-3c1e-4daa-bb56-8d8a1e5b4770"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:42:41.793247 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:41.793147 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6158043c-3c1e-4daa-bb56-8d8a1e5b4770-maas-api-service-ca\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:42:42.477536 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:42.477503 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" Jul 23 17:42:42.477763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:42.477524 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b" event={"ID":"6158043c-3c1e-4daa-bb56-8d8a1e5b4770","Type":"ContainerDied","Data":"412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf"} Jul 23 17:42:42.477763 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:42.477677 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="412398a6ca8cab1758c53e4c3e1b9c6fc00eea1aac8f234cc1a3edbdf7dc98cf" Jul 23 17:42:53.949228 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:53.949177 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b"] Jul 23 17:42:53.952032 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:53.951996 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-219aa1-ms65b"] Jul 23 17:42:55.132507 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:42:55.132461 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6158043c-3c1e-4daa-bb56-8d8a1e5b4770" path="/var/lib/kubelet/pods/6158043c-3c1e-4daa-bb56-8d8a1e5b4770/volumes" Jul 23 17:43:02.318638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:02.318596 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:43:02.319072 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:02.318870 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" containerID="cri-o://71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45" gracePeriod=30 Jul 23 17:43:07.587709 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.587616 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:43:07.616909 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.616870 2584 generic.go:356] "Generic (PLEG): container finished" podID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerID="71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45" exitCode=0 Jul 23 17:43:07.617099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.616940 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" Jul 23 17:43:07.617099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.616956 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerDied","Data":"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45"} Jul 23 17:43:07.617099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.617017 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74" event={"ID":"c1204be6-d7a7-4696-88f6-35decfadc1cc","Type":"ContainerDied","Data":"0770bc16e0b6abe28b020bf682db6888b90264204a31a97a3992f5253b9c7c0c"} Jul 23 17:43:07.617099 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.617046 2584 scope.go:117] "RemoveContainer" containerID="71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45" Jul 23 17:43:07.628682 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.628660 2584 scope.go:117] "RemoveContainer" containerID="4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2" Jul 23 17:43:07.640246 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.640218 2584 scope.go:117] "RemoveContainer" containerID="71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45" Jul 23 17:43:07.640716 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:43:07.640691 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45\": container with ID starting with 71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45 not found: ID does not exist" containerID="71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45" Jul 23 17:43:07.640788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.640728 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45"} err="failed to get container status \"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45\": rpc error: code = NotFound desc = could not find container \"71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45\": container with ID starting with 71e2d5ea003dd85704f5ed283080fc8bdf9b53a07561f56efb2245e18f8cac45 not found: ID does not exist" Jul 23 17:43:07.640788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.640748 2584 scope.go:117] "RemoveContainer" containerID="4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2" Jul 23 17:43:07.641015 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:43:07.640984 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2\": container with ID starting with 4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2 not found: ID does not exist" containerID="4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2" Jul 23 17:43:07.641075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.641025 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2"} err="failed to get container status \"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2\": rpc error: code = NotFound desc = could not find container \"4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2\": container with ID starting with 4e0fa6d386ef07adb76276b9ad25f85e6ca0cf918a6cd1fc4f3ddd3e4ad57dd2 not found: ID does not exist" Jul 23 17:43:07.651598 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651564 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651632 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw5hx\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651668 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651692 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651746 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651777 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.651813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.651804 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.652121 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.652058 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:07.652182 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.652170 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.652420 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.652396 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:43:07.652713 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.652686 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:07.654036 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.654002 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx" (OuterVolumeSpecName: "kube-api-access-jw5hx") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "kube-api-access-jw5hx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:43:07.654136 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.654040 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:43:07.654314 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.654285 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:43:07.654438 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.654293 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:07.752589 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752543 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.752589 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752586 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket\") pod \"c1204be6-d7a7-4696-88f6-35decfadc1cc\" (UID: \"c1204be6-d7a7-4696-88f6-35decfadc1cc\") " Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752789 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752799 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752808 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752816 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1204be6-d7a7-4696-88f6-35decfadc1cc-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752824 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.752846 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752834 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw5hx\" (UniqueName: \"kubernetes.io/projected/c1204be6-d7a7-4696-88f6-35decfadc1cc-kube-api-access-jw5hx\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.753122 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752901 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:07.753122 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.752911 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1204be6-d7a7-4696-88f6-35decfadc1cc" (UID: "c1204be6-d7a7-4696-88f6-35decfadc1cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:07.853810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.853697 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.853810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.853751 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1204be6-d7a7-4696-88f6-35decfadc1cc-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:07.947380 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.947323 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:43:07.952807 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:07.952755 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-219aa1-openshift-default-56b4974cd7-l2k74"] Jul 23 17:43:09.131781 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:09.131744 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" path="/var/lib/kubelet/pods/c1204be6-d7a7-4696-88f6-35decfadc1cc/volumes" Jul 23 17:43:24.886672 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:24.886636 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:43:24.887126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:24.886870 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" containerID="cri-o://69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503" gracePeriod=30 Jul 23 17:43:25.989509 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.989474 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.989866 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.989878 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.989895 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6158043c-3c1e-4daa-bb56-8d8a1e5b4770" containerName="revoke-keys" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.989903 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6158043c-3c1e-4daa-bb56-8d8a1e5b4770" containerName="revoke-keys" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.990000 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6158043c-3c1e-4daa-bb56-8d8a1e5b4770" containerName="revoke-keys" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.990008 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:25.990022 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.990015 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:25.993289 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.993269 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:25.999630 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:25.999605 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f9b147-openshift-default-dockercfg-dpng7\"" Jul 23 17:43:26.035734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.035697 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:43:26.136386 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136317 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136406 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwqjg\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136479 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136504 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136531 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136615 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136597 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.136879 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.136652 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237211 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237211 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237261 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237304 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237391 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237828 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237531 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqjg\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237828 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237731 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.237828 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.237743 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.238071 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.238042 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.238252 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.238226 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.238325 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.238190 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.239941 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.239869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.240075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.239982 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.247438 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.247408 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.247773 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.247750 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqjg\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg\") pod \"e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.304493 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.304451 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:26.482263 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.482235 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:43:26.484144 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:43:26.484115 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb68c39d_5d7d_4ca9_9fbf_6f839e79e89b.slice/crio-50ceada9b5649ecc1743e00c98810cf2ddf8cbf3442957725f8bd3bc1626c377 WatchSource:0}: Error finding container 50ceada9b5649ecc1743e00c98810cf2ddf8cbf3442957725f8bd3bc1626c377: Status 404 returned error can't find the container with id 50ceada9b5649ecc1743e00c98810cf2ddf8cbf3442957725f8bd3bc1626c377 Jul 23 17:43:26.486735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.486701 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:26.486809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.486776 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:26.486809 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.486805 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:26.721173 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.721126 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerStarted","Data":"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e"} Jul 23 17:43:26.721407 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.721192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerStarted","Data":"50ceada9b5649ecc1743e00c98810cf2ddf8cbf3442957725f8bd3bc1626c377"} Jul 23 17:43:26.756417 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:26.756266 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podStartSLOduration=1.756241255 podStartE2EDuration="1.756241255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:43:26.753778995 +0000 UTC m=+2328.107640411" watchObservedRunningTime="2026-07-23 17:43:26.756241255 +0000 UTC m=+2328.110102636" Jul 23 17:43:27.305165 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:27.305123 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:27.310498 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:27.310472 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:27.726637 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:27.726600 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:27.727586 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:27.727562 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:30.174873 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.174844 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:43:30.279555 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279514 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.279748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279577 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.279748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279625 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.279748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279674 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp2wj\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.279748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279702 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.279748 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279735 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.280003 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279871 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.280003 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279936 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.280003 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.279972 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket\") pod \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\" (UID: \"f5905a0a-d597-4210-ac8f-cfbaa68e51d1\") " Jul 23 17:43:30.280160 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280032 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:43:30.280160 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280047 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:30.280160 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280068 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:30.280325 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280292 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:30.280518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280474 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.280518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280500 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.280518 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280516 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.280935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280529 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.280935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.280901 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:30.282396 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.282350 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:43:30.282497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.282391 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:43:30.282623 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.282600 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj" (OuterVolumeSpecName: "kube-api-access-pp2wj") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "kube-api-access-pp2wj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:43:30.282946 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.282928 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5905a0a-d597-4210-ac8f-cfbaa68e51d1" (UID: "f5905a0a-d597-4210-ac8f-cfbaa68e51d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:43:30.381717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.381679 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.381717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.381709 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.381717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.381723 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.381974 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.381734 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp2wj\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-kube-api-access-pp2wj\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.381974 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.381743 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5905a0a-d597-4210-ac8f-cfbaa68e51d1-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:43:30.743028 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.742995 2584 generic.go:356] "Generic (PLEG): container finished" podID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerID="69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503" exitCode=0 Jul 23 17:43:30.743208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.743048 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" Jul 23 17:43:30.743208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.743057 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerDied","Data":"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503"} Jul 23 17:43:30.743208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.743100 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7" event={"ID":"f5905a0a-d597-4210-ac8f-cfbaa68e51d1","Type":"ContainerDied","Data":"dcffa6d8fac0e10312748a23862d01046366ccc240effe9682201e58acfc4db6"} Jul 23 17:43:30.743208 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.743124 2584 scope.go:117] "RemoveContainer" containerID="69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503" Jul 23 17:43:30.752875 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.752856 2584 scope.go:117] "RemoveContainer" containerID="aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06" Jul 23 17:43:30.761768 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.761744 2584 scope.go:117] "RemoveContainer" containerID="69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503" Jul 23 17:43:30.762078 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:43:30.762058 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503\": container with ID starting with 69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503 not found: ID does not exist" containerID="69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503" Jul 23 17:43:30.762149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.762088 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503"} err="failed to get container status \"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503\": rpc error: code = NotFound desc = could not find container \"69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503\": container with ID starting with 69598b2c1c106b37b223b381ea9729d602ab736660c2d38c909dc43b3778d503 not found: ID does not exist" Jul 23 17:43:30.762149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.762106 2584 scope.go:117] "RemoveContainer" containerID="aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06" Jul 23 17:43:30.762336 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:43:30.762317 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06\": container with ID starting with aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06 not found: ID does not exist" containerID="aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06" Jul 23 17:43:30.762411 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.762340 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06"} err="failed to get container status \"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06\": rpc error: code = NotFound desc = could not find container \"aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06\": container with ID starting with aba6cdd8799dae91077d6b0c8cbd949e319f82ef64e79b96375a9ac9045a9a06 not found: ID does not exist" Jul 23 17:43:30.773414 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.773381 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:43:30.781876 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:30.781846 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7363d9-openshift-default-66d7c85995-pqdz7"] Jul 23 17:43:31.131808 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:31.131686 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" path="/var/lib/kubelet/pods/f5905a0a-d597-4210-ac8f-cfbaa68e51d1/volumes" Jul 23 17:43:34.372838 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.372800 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:43:34.373264 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373249 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:34.373320 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373266 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1204be6-d7a7-4696-88f6-35decfadc1cc" containerName="istio-proxy" Jul 23 17:43:34.373320 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373275 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:43:34.373320 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373281 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:43:34.373456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373384 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:43:34.373456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.373401 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:43:34.378083 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.378058 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.381946 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.381919 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6f74dd-openshift-default-dockercfg-jzgj7\"" Jul 23 17:43:34.391412 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.391378 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:43:34.519514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519471 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519514 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519521 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519597 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519652 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519679 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519749 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519793 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519788 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519994 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519874 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.519994 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.519903 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrdng\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621197 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621197 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621526 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621262 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621526 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621306 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrdng\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621526 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621526 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621511 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621596 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621735 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621624 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621878 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621777 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.621878 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.621857 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.622043 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.622024 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.622102 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.622066 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.622149 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.622096 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.623974 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.623910 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.624180 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.624163 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.637315 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.637276 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.637477 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.637415 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrdng\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng\") pod \"e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.689983 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.689932 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:34.863395 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.863347 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:43:34.864431 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:43:34.864406 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a38dbe_db53_47f6_a35d_ed2d85da9b4d.slice/crio-eea748942d812f66dc943cbc9bb245fe21522f4de6ac1a69f7e54a43508fa4cb WatchSource:0}: Error finding container eea748942d812f66dc943cbc9bb245fe21522f4de6ac1a69f7e54a43508fa4cb: Status 404 returned error can't find the container with id eea748942d812f66dc943cbc9bb245fe21522f4de6ac1a69f7e54a43508fa4cb Jul 23 17:43:34.866622 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.866587 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:34.866707 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.866660 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:34.866707 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:34.866690 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:35.775068 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:35.775034 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerStarted","Data":"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2"} Jul 23 17:43:35.775486 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:35.775087 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerStarted","Data":"eea748942d812f66dc943cbc9bb245fe21522f4de6ac1a69f7e54a43508fa4cb"} Jul 23 17:43:35.808622 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:35.808566 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podStartSLOduration=1.808548142 podStartE2EDuration="1.808548142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:43:35.803295068 +0000 UTC m=+2337.157156448" watchObservedRunningTime="2026-07-23 17:43:35.808548142 +0000 UTC m=+2337.162409523" Jul 23 17:43:36.690691 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:36.690649 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:36.695706 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:36.695675 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:36.780546 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:36.780503 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:36.781488 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:36.781467 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:43.767605 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:43.767546 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 17:43:43.768051 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:43.767629 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 17:43:44.843128 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.843032 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerID="1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e" exitCode=0 Jul 23 17:43:44.843128 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.843111 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerDied","Data":"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e"} Jul 23 17:43:44.843746 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.843729 2584 scope.go:117] "RemoveContainer" containerID="1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e" Jul 23 17:43:44.845042 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.845006 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:44.845156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.845086 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:44.845156 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:44.845127 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:45.854544 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:45.854505 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerStarted","Data":"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c"} Jul 23 17:43:46.304934 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:46.304621 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:46.305306 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:46.305281 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 23 17:43:46.305500 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:46.305473 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 23 17:43:47.305554 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:47.305506 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 23 17:43:47.306108 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:47.305580 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 23 17:43:48.305582 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:48.305531 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 23 17:43:48.306126 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:48.305604 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 23 17:43:49.305948 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:49.305901 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 23 17:43:49.306653 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:49.306609 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 23 17:43:50.309495 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:50.309455 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:50.309956 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:50.309748 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:50.310856 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:50.310837 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:43:51.782642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.782609 2584 patch_prober.go:28] interesting pod/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 23 17:43:51.783090 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.782667 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 23 17:43:51.892439 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.892314 2584 generic.go:356] "Generic (PLEG): container finished" podID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerID="93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2" exitCode=0 Jul 23 17:43:51.892439 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.892391 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerDied","Data":"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2"} Jul 23 17:43:51.892935 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.892915 2584 scope.go:117] "RemoveContainer" containerID="93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2" Jul 23 17:43:51.894098 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.894070 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:51.894214 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.894139 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:51.894214 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:51.894191 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:43:52.906729 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:52.906694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerStarted","Data":"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db"} Jul 23 17:43:53.690473 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:53.690423 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:53.690711 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:53.690688 2584 patch_prober.go:28] interesting pod/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 23 17:43:53.690766 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:53.690737 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 23 17:43:54.690631 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:54.690587 2584 patch_prober.go:28] interesting pod/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 23 17:43:54.691139 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:54.690658 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 23 17:43:55.691205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:55.691166 2584 patch_prober.go:28] interesting pod/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 23 17:43:55.691732 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:55.691225 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 23 17:43:56.690556 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:56.690514 2584 patch_prober.go:28] interesting pod/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 23 17:43:56.690749 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:56.690586 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 23 17:43:57.694870 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:57.694841 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:57.695385 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:57.695132 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:43:57.695988 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:43:57.695970 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:44:18.213272 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:18.213229 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:44:18.213843 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:18.213536 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" containerID="cri-o://bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c" gracePeriod=30 Jul 23 17:44:20.312309 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:20.312261 2584 patch_prober.go:28] interesting pod/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 17:44:20.312743 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:20.312343 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 17:44:23.476023 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.475944 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:44:23.517947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.517902 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.517947 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.517947 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.517988 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwqjg\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518027 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518100 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518123 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518150 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518180 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518207 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert\") pod \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\" (UID: \"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b\") " Jul 23 17:44:23.518221 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518213 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:23.518680 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518524 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.519012 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.518963 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:23.519147 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.519084 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:23.519285 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.519260 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:23.519438 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.519407 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:44:23.520841 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.520813 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:44:23.521065 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.521032 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:44:23.521176 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.521144 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg" (OuterVolumeSpecName: "kube-api-access-gwqjg") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "kube-api-access-gwqjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:44:23.521227 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.521195 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" (UID: "fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:23.619436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619398 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619430 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwqjg\" (UniqueName: \"kubernetes.io/projected/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-kube-api-access-gwqjg\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619441 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619451 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619460 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619467 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619475 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:23.619689 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:23.619483 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:24.087320 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.087282 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerID="bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c" exitCode=0 Jul 23 17:44:24.087531 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.087335 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" Jul 23 17:44:24.087531 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.087398 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerDied","Data":"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c"} Jul 23 17:44:24.087531 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.087455 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc" event={"ID":"fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b","Type":"ContainerDied","Data":"50ceada9b5649ecc1743e00c98810cf2ddf8cbf3442957725f8bd3bc1626c377"} Jul 23 17:44:24.087531 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.087485 2584 scope.go:117] "RemoveContainer" containerID="bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c" Jul 23 17:44:24.097045 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.097025 2584 scope.go:117] "RemoveContainer" containerID="1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e" Jul 23 17:44:24.105970 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.105948 2584 scope.go:117] "RemoveContainer" containerID="bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c" Jul 23 17:44:24.106276 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:44:24.106256 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c\": container with ID starting with bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c not found: ID does not exist" containerID="bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c" Jul 23 17:44:24.106350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.106289 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c"} err="failed to get container status \"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c\": rpc error: code = NotFound desc = could not find container \"bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c\": container with ID starting with bb9bdbdbb53f2a4dda36b63f9a8e87baa362dc70ba1a53f4114f20910c64d05c not found: ID does not exist" Jul 23 17:44:24.106350 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.106311 2584 scope.go:117] "RemoveContainer" containerID="1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e" Jul 23 17:44:24.106590 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:44:24.106575 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e\": container with ID starting with 1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e not found: ID does not exist" containerID="1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e" Jul 23 17:44:24.106644 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.106592 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e"} err="failed to get container status \"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e\": rpc error: code = NotFound desc = could not find container \"1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e\": container with ID starting with 1abbd1a25a3ac8f59efddc86107760e9f2de066a7f93cdbe55aa8bc27479541e not found: ID does not exist" Jul 23 17:44:24.111068 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.111032 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:44:24.115394 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:24.115342 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f9b147-openshift-default-7865c9889c-7l2sc"] Jul 23 17:44:25.131744 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:25.131707 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" path="/var/lib/kubelet/pods/fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b/volumes" Jul 23 17:44:39.243272 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:39.243241 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:44:39.246516 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:39.246490 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:44:46.225902 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:46.225865 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:44:46.226373 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:46.226101 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" containerID="cri-o://d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db" gracePeriod=30 Jul 23 17:44:47.223192 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223149 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:44:47.223573 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223560 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:44:47.223677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223575 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5905a0a-d597-4210-ac8f-cfbaa68e51d1" containerName="istio-proxy" Jul 23 17:44:47.223677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223593 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.223677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223609 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.223677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223628 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.223677 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223633 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.223853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223717 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.223853 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.223726 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb68c39d-5d7d-4ca9-9fbf-6f839e79e89b" containerName="istio-proxy" Jul 23 17:44:47.226968 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.226948 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.229885 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.229861 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b9d590-openshift-default-dockercfg-t6mng\"" Jul 23 17:44:47.239260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.239229 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:44:47.355767 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355726 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355767 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355772 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355802 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355820 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355875 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r44l6\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355924 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.355995 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.355944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457353 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457304 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457353 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457391 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457438 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457477 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r44l6\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457588 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457981 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457861 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.457981 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.457942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.458089 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.458004 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.458142 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.458089 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.458639 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.458612 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.460155 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.460128 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.460242 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.460226 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.467075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.467038 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.467075 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.467071 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r44l6\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6\") pod \"e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.538753 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.538656 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:47.698456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.698426 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:44:47.701136 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:44:47.701104 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7844ed9_cfcf_483e_a470_8e7caf47a7c0.slice/crio-83806bbbea73d9139a3331526e17c638ea304f03c533fba7c23e646a6288337b WatchSource:0}: Error finding container 83806bbbea73d9139a3331526e17c638ea304f03c533fba7c23e646a6288337b: Status 404 returned error can't find the container with id 83806bbbea73d9139a3331526e17c638ea304f03c533fba7c23e646a6288337b Jul 23 17:44:47.703475 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.703434 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:47.703581 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.703512 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:47.703581 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:47.703547 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:48.216338 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:48.216296 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerStarted","Data":"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b"} Jul 23 17:44:48.216626 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:48.216387 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerStarted","Data":"83806bbbea73d9139a3331526e17c638ea304f03c533fba7c23e646a6288337b"} Jul 23 17:44:48.268200 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:48.268141 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podStartSLOduration=1.268123164 podStartE2EDuration="1.268123164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:44:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:44:48.267927123 +0000 UTC m=+2409.621788505" watchObservedRunningTime="2026-07-23 17:44:48.268123164 +0000 UTC m=+2409.621984543" Jul 23 17:44:48.539701 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:48.539581 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:48.544770 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:48.544739 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:49.222056 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:49.222025 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:49.223193 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:49.223173 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:44:51.499420 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.499318 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:44:51.601782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.601750 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.601782 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.601782 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrdng\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.602057 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.601855 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.602264 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602243 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602280 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602343 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602417 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602447 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602483 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo\") pod \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\" (UID: \"03a38dbe-db53-47f6-a35d-ed2d85da9b4d\") " Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602663 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602800 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602930 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.603017 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.602950 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.603283 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:44:51.604202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.604043 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:51.605658 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.605596 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:44:51.607322 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.607290 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng" (OuterVolumeSpecName: "kube-api-access-wrdng") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "kube-api-access-wrdng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:44:51.607529 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.607505 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:44:51.607821 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.607792 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03a38dbe-db53-47f6-a35d-ed2d85da9b4d" (UID: "03a38dbe-db53-47f6-a35d-ed2d85da9b4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:44:51.704588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704550 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704579 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704587 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704595 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704588 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704602 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704610 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:51.704917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:51.704618 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrdng\" (UniqueName: \"kubernetes.io/projected/03a38dbe-db53-47f6-a35d-ed2d85da9b4d-kube-api-access-wrdng\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:44:52.247173 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.247140 2584 generic.go:356] "Generic (PLEG): container finished" podID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerID="d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db" exitCode=0 Jul 23 17:44:52.247393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.247191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerDied","Data":"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db"} Jul 23 17:44:52.247393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.247202 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" Jul 23 17:44:52.247393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.247260 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw" event={"ID":"03a38dbe-db53-47f6-a35d-ed2d85da9b4d","Type":"ContainerDied","Data":"eea748942d812f66dc943cbc9bb245fe21522f4de6ac1a69f7e54a43508fa4cb"} Jul 23 17:44:52.247393 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.247283 2584 scope.go:117] "RemoveContainer" containerID="d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db" Jul 23 17:44:52.257717 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.257697 2584 scope.go:117] "RemoveContainer" containerID="93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2" Jul 23 17:44:52.267258 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.267235 2584 scope.go:117] "RemoveContainer" containerID="d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db" Jul 23 17:44:52.267592 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:44:52.267571 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db\": container with ID starting with d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db not found: ID does not exist" containerID="d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db" Jul 23 17:44:52.267669 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.267614 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db"} err="failed to get container status \"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db\": rpc error: code = NotFound desc = could not find container \"d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db\": container with ID starting with d2a7726c1ddc8ae8835d8e36a80a66e9734dba9993eef192381d09374c4778db not found: ID does not exist" Jul 23 17:44:52.267669 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.267653 2584 scope.go:117] "RemoveContainer" containerID="93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2" Jul 23 17:44:52.267941 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:44:52.267922 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2\": container with ID starting with 93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2 not found: ID does not exist" containerID="93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2" Jul 23 17:44:52.267991 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.267947 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2"} err="failed to get container status \"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2\": rpc error: code = NotFound desc = could not find container \"93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2\": container with ID starting with 93ec43cc5408152499cf6f8678e9341e73294a6724e3913e93e5da501350ecb2 not found: ID does not exist" Jul 23 17:44:52.272280 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.272241 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:44:52.281009 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:52.280974 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6f74dd-openshift-default-b547d8dd7-hprfw"] Jul 23 17:44:53.130977 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:53.130931 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" path="/var/lib/kubelet/pods/03a38dbe-db53-47f6-a35d-ed2d85da9b4d/volumes" Jul 23 17:44:55.482940 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.482906 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:44:55.483491 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.483471 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:44:55.483575 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.483493 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:44:55.483680 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.483666 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:44:55.483738 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.483690 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:44:55.488202 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.488174 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.491687 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.491662 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-9c4020-openshift-default-dockercfg-szxd6\"" Jul 23 17:44:55.504788 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.504754 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:44:55.642301 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642308 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642439 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642478 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t225b\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642527 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642507 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642574 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642632 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642651 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.642755 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.642671 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743506 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t225b\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743658 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743812 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.743943 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.743841 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.744224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.744006 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.744224 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.744116 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.744330 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.744248 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.744398 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.744347 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.744647 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.744629 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.746115 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.746093 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.746760 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.746735 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.752277 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.752237 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.752508 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.752482 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t225b\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b\") pod \"e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.799402 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.799336 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:55.960917 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.960882 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:44:55.962112 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:44:55.962085 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d59910_55e1_4fb8_99ec_da17089a960f.slice/crio-d91d535e0e371ab81c72d50cd9b9fbaf38c90246c1450ed9a089d8ad4f0a555c WatchSource:0}: Error finding container d91d535e0e371ab81c72d50cd9b9fbaf38c90246c1450ed9a089d8ad4f0a555c: Status 404 returned error can't find the container with id d91d535e0e371ab81c72d50cd9b9fbaf38c90246c1450ed9a089d8ad4f0a555c Jul 23 17:44:55.964513 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.964469 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:55.964623 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.964566 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:55.964623 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:55.964615 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:44:56.271734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:56.271614 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerStarted","Data":"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c"} Jul 23 17:44:56.271734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:56.271694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerStarted","Data":"d91d535e0e371ab81c72d50cd9b9fbaf38c90246c1450ed9a089d8ad4f0a555c"} Jul 23 17:44:56.800118 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:56.800077 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:57.805655 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:57.805618 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:57.831874 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:57.831808 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podStartSLOduration=2.831784673 podStartE2EDuration="2.831784673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:44:56.320455556 +0000 UTC m=+2417.674316935" watchObservedRunningTime="2026-07-23 17:44:57.831784673 +0000 UTC m=+2419.185646055" Jul 23 17:44:58.289031 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:58.289004 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:44:58.289912 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:44:58.289892 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:45:05.263613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:05.263556 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 17:45:05.264123 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:05.263673 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 17:45:06.579405 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:45:06.579337 2584 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7844ed9_cfcf_483e_a470_8e7caf47a7c0.slice/crio-ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7844ed9_cfcf_483e_a470_8e7caf47a7c0.slice/crio-conmon-ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b.scope\": RecentStats: unable to find data in memory cache]" Jul 23 17:45:07.347967 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.347930 2584 generic.go:356] "Generic (PLEG): container finished" podID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerID="ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b" exitCode=0 Jul 23 17:45:07.348147 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.347981 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerDied","Data":"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b"} Jul 23 17:45:07.348463 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.348444 2584 scope.go:117] "RemoveContainer" containerID="ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b" Jul 23 17:45:07.349635 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.349601 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:07.349751 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.349711 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:07.349819 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:07.349764 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:08.356283 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:08.356244 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerStarted","Data":"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada"} Jul 23 17:45:08.539613 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:08.539567 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:45:08.540049 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:08.540022 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 23 17:45:08.540260 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:08.540232 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 23 17:45:09.539552 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:09.539509 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 23 17:45:09.540227 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:09.540187 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 23 17:45:10.539598 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:10.539504 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 23 17:45:10.540000 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:10.539618 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 23 17:45:11.539436 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:11.539391 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 23 17:45:11.539688 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:11.539472 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 23 17:45:12.540039 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:12.540000 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 23 17:45:12.540552 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:12.540080 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 23 17:45:13.543279 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:13.543249 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:45:13.543810 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:13.543570 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:45:13.544340 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:13.544317 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:45:13.664802 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:13.663634 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:52782->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 23 17:45:13.664802 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:13.663721 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:52782->10.132.0.56:15021: read: connection reset by peer" Jul 23 17:45:14.391114 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.391083 2584 generic.go:356] "Generic (PLEG): container finished" podID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerID="cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c" exitCode=0 Jul 23 17:45:14.391300 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.391157 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerDied","Data":"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c"} Jul 23 17:45:14.391718 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.391699 2584 scope.go:117] "RemoveContainer" containerID="cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c" Jul 23 17:45:14.392828 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.392795 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:14.392926 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.392908 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:14.392971 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:14.392957 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 17:45:15.403372 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:15.403311 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerStarted","Data":"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b"} Jul 23 17:45:15.799973 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:15.799930 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:45:15.800287 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:15.800256 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 23 17:45:15.800399 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:15.800324 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 23 17:45:16.800467 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:16.800419 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 23 17:45:16.801005 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:16.800493 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 23 17:45:17.800185 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:17.800130 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 23 17:45:17.800442 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:17.800209 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 23 17:45:18.799900 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:18.799849 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 23 17:45:18.800487 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:18.799945 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 23 17:45:19.804205 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:19.804173 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:45:19.804638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:19.804473 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:45:19.805188 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:19.805169 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:45:39.368487 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:45:39.368456 2584 scope.go:117] "RemoveContainer" containerID="0043ca2014b5042d3ee7686d8e84258fd9d02168a2039a0d461e18c2ee499f0f" Jul 23 17:48:39.408456 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:48:39.408421 2584 scope.go:117] "RemoveContainer" containerID="8401131e8579695f664c19b6264e3d2dcee705c4ef26403e92e637036a1339ff" Jul 23 17:49:39.279629 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:49:39.279543 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:49:39.282694 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:49:39.282669 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:51:23.639641 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:23.639606 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:51:23.640162 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:23.639867 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" containerID="cri-o://cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada" gracePeriod=30 Jul 23 17:51:28.546496 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.546459 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 17:51:28.546889 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.546518 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 17:51:28.904642 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.904613 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:51:28.927796 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927762 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r44l6\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927844 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927869 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927906 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927930 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927955 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.927996 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.927986 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.928302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928014 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.928302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928074 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket\") pod \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\" (UID: \"f7844ed9-cfcf-483e-a470-8e7caf47a7c0\") " Jul 23 17:51:28.928302 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928154 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:28.928489 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928413 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:51:28.928489 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928443 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:28.929422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.928716 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:28.929422 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.929213 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:28.929593 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.929567 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:28.930476 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.930448 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:28.930599 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.930539 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:51:28.930666 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.930592 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:51:28.931597 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:28.931574 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6" (OuterVolumeSpecName: "kube-api-access-r44l6") pod "f7844ed9-cfcf-483e-a470-8e7caf47a7c0" (UID: "f7844ed9-cfcf-483e-a470-8e7caf47a7c0"). InnerVolumeSpecName "kube-api-access-r44l6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:51:29.029789 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029748 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r44l6\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-kube-api-access-r44l6\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.029789 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029782 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.029789 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029792 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.030035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029801 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.030035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029809 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.030035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029816 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.030035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029823 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.030035 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.029830 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7844ed9-cfcf-483e-a470-8e7caf47a7c0-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:29.365991 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.365960 2584 generic.go:356] "Generic (PLEG): container finished" podID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerID="cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada" exitCode=0 Jul 23 17:51:29.366179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.366011 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" Jul 23 17:51:29.366179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.366046 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerDied","Data":"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada"} Jul 23 17:51:29.366179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.366105 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw" event={"ID":"f7844ed9-cfcf-483e-a470-8e7caf47a7c0","Type":"ContainerDied","Data":"83806bbbea73d9139a3331526e17c638ea304f03c533fba7c23e646a6288337b"} Jul 23 17:51:29.366179 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.366128 2584 scope.go:117] "RemoveContainer" containerID="cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada" Jul 23 17:51:29.375644 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.375621 2584 scope.go:117] "RemoveContainer" containerID="ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b" Jul 23 17:51:29.385055 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.385032 2584 scope.go:117] "RemoveContainer" containerID="cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada" Jul 23 17:51:29.385414 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:51:29.385386 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada\": container with ID starting with cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada not found: ID does not exist" containerID="cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada" Jul 23 17:51:29.385497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.385428 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada"} err="failed to get container status \"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada\": rpc error: code = NotFound desc = could not find container \"cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada\": container with ID starting with cf7169133d747da7a9dc66fa1ba3079af01ccce6d059c77090319f1854b1eada not found: ID does not exist" Jul 23 17:51:29.385497 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.385452 2584 scope.go:117] "RemoveContainer" containerID="ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b" Jul 23 17:51:29.385745 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:51:29.385720 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b\": container with ID starting with ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b not found: ID does not exist" containerID="ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b" Jul 23 17:51:29.385878 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.385751 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b"} err="failed to get container status \"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b\": rpc error: code = NotFound desc = could not find container \"ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b\": container with ID starting with ffe9d15427380805fda35205fec35f5e43afd8fbc79ac838c23b6934deb89d5b not found: ID does not exist" Jul 23 17:51:29.387085 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.387061 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:51:29.389172 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:29.389117 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b9d590-openshift-default-6bd7cb86ff-xdxrw"] Jul 23 17:51:31.131640 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:31.131600 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" path="/var/lib/kubelet/pods/f7844ed9-cfcf-483e-a470-8e7caf47a7c0/volumes" Jul 23 17:51:46.111528 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:46.111492 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:51:46.112143 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:46.111737 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" containerID="cri-o://0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b" gracePeriod=30 Jul 23 17:51:49.807297 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:49.807257 2584 patch_prober.go:28] interesting pod/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 17:51:49.807731 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:49.807324 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 17:51:51.375874 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.375799 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:51:51.445183 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445148 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t225b\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445183 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445183 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445484 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445226 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445484 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445341 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445484 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445426 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445537 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445589 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445616 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445622 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:51.445848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445648 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data\") pod \"b1d59910-55e1-4fb8-99ec-da17089a960f\" (UID: \"b1d59910-55e1-4fb8-99ec-da17089a960f\") " Jul 23 17:51:51.445848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445641 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:51.445848 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445824 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:51.446008 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.445922 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 17:51:51.446062 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.446018 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-certs\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.446062 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.446036 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-workload-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.446062 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.446049 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-credential-socket\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.446486 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.446466 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:51.447512 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.447491 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 17:51:51.447612 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.447519 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:51:51.447790 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.447770 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b" (OuterVolumeSpecName: "kube-api-access-t225b") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "kube-api-access-t225b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 17:51:51.448081 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.448063 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1d59910-55e1-4fb8-99ec-da17089a960f" (UID: "b1d59910-55e1-4fb8-99ec-da17089a960f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 17:51:51.478269 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.478235 2584 generic.go:356] "Generic (PLEG): container finished" podID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerID="0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b" exitCode=0 Jul 23 17:51:51.478460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.478289 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" Jul 23 17:51:51.478460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.478320 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerDied","Data":"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b"} Jul 23 17:51:51.478460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.478397 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82" event={"ID":"b1d59910-55e1-4fb8-99ec-da17089a960f","Type":"ContainerDied","Data":"d91d535e0e371ab81c72d50cd9b9fbaf38c90246c1450ed9a089d8ad4f0a555c"} Jul 23 17:51:51.478460 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.478422 2584 scope.go:117] "RemoveContainer" containerID="0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b" Jul 23 17:51:51.491228 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.491213 2584 scope.go:117] "RemoveContainer" containerID="cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c" Jul 23 17:51:51.500352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.500232 2584 scope.go:117] "RemoveContainer" containerID="0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b" Jul 23 17:51:51.500670 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:51:51.500646 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b\": container with ID starting with 0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b not found: ID does not exist" containerID="0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b" Jul 23 17:51:51.500764 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.500684 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b"} err="failed to get container status \"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b\": rpc error: code = NotFound desc = could not find container \"0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b\": container with ID starting with 0b5230915bf81c83795f0d5f654d1cb693a1b61c270af50a1d61a7b815e56a7b not found: ID does not exist" Jul 23 17:51:51.500764 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.500710 2584 scope.go:117] "RemoveContainer" containerID="cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c" Jul 23 17:51:51.500986 ip-10-0-136-29 kubenswrapper[2584]: E0723 17:51:51.500969 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c\": container with ID starting with cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c not found: ID does not exist" containerID="cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c" Jul 23 17:51:51.501025 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.500991 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c"} err="failed to get container status \"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c\": rpc error: code = NotFound desc = could not find container \"cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c\": container with ID starting with cc59bb52344161c5a56acae730151ece968e5c57f8c26cd303c122ae7d95723c not found: ID does not exist" Jul 23 17:51:51.502074 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.502050 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:51:51.507238 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.507211 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-9c4020-openshift-default-9d7c7bdb7-vdd82"] Jul 23 17:51:51.546595 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546565 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-podinfo\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.546595 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546591 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-data\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.546595 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546600 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t225b\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-kube-api-access-t225b\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.546813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546609 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-token\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.546813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546618 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1d59910-55e1-4fb8-99ec-da17089a960f-istio-envoy\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:51.546813 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:51.546625 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1d59910-55e1-4fb8-99ec-da17089a960f-istiod-ca-cert\") on node \"ip-10-0-136-29.ec2.internal\" DevicePath \"\"" Jul 23 17:51:52.788316 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:52.788291 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f96g6_180c167e-3fa3-4214-b2e5-74796644b1d5/server/0.log" Jul 23 17:51:53.131906 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:53.131820 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" path="/var/lib/kubelet/pods/b1d59910-55e1-4fb8-99ec-da17089a960f/volumes" Jul 23 17:51:53.157840 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:53.157808 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9b469455-qb96r_16087e24-40c7-44f8-8e2d-a683abb54b27/copy-manifests/0.log" Jul 23 17:51:53.164343 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:53.164321 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9b469455-qb96r_16087e24-40c7-44f8-8e2d-a683abb54b27/manager/0.log" Jul 23 17:51:53.751600 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:53.751573 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747115-hs8cz_6cd1423c-8992-4ce5-a070-3ee67217cc2c/cleanup/0.log" Jul 23 17:51:53.870125 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:53.870095 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747130-68fgf_89eff329-ccb4-4ea8-82b9-0f1076ee01e2/cleanup/0.log" Jul 23 17:51:55.545413 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:55.545350 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dg5xc_8d8a8d05-4c08-4adf-8a7e-6856df120537/manager/0.log" Jul 23 17:51:56.658038 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:56.658004 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sw65h_132969f0-927a-4c64-b0ba-321a5a51f30d/istio-proxy/0.log" Jul 23 17:51:56.784747 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:56.784713 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rglvp_1f8fb21c-fc35-40ae-8ee8-b74f4bae46da/discovery/0.log" Jul 23 17:51:56.908638 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:56.908547 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7466c75dcd-bl6q4_2c79f73f-844e-4588-94b9-77f13a8a2086/kube-auth-proxy/0.log" Jul 23 17:51:57.148411 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:57.148379 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/1.log" Jul 23 17:51:57.276508 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:57.276476 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-47hhw_1c4f0d2f-57ca-43f4-93c5-b2229161fde6/payload-pre-processing/0.log" Jul 23 17:51:57.511827 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:57.511798 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f9cf5b5b-kqk57_3021ee46-d3b1-4152-bec2-24e89cabe038/router/0.log" Jul 23 17:51:57.867397 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:57.867369 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xms79_6ce325d2-91bb-4af1-b3a6-30013a90abfe/main/0.log" Jul 23 17:51:57.992109 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:51:57.992065 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nxwqs_f1e56f08-742b-4e50-8fa8-47adf287e465/main/0.log" Jul 23 17:52:07.634270 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:07.634226 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g42dl_403649e1-c982-4cb0-9868-4b60e570db92/global-pull-secret-syncer/0.log" Jul 23 17:52:07.730294 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:07.730263 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sl89n_7188a0ac-f5c3-4a35-a37a-c504a176efe0/konnectivity-agent/0.log" Jul 23 17:52:07.852194 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:07.852161 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-29.ec2.internal_e406453e6e016770b211c4662b0c1b45/haproxy/0.log" Jul 23 17:52:12.492152 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:12.492111 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dg5xc_8d8a8d05-4c08-4adf-8a7e-6856df120537/manager/0.log" Jul 23 17:52:14.977587 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:14.977552 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-k98pv_3582e243-6f99-400c-b7e4-e4847ba68e9b/cluster-monitoring-operator/0.log" Jul 23 17:52:15.420174 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:15.420090 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4dzk_538386f2-bb08-4d4b-8445-84ca1f37b434/node-exporter/0.log" Jul 23 17:52:15.459618 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:15.459584 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4dzk_538386f2-bb08-4d4b-8445-84ca1f37b434/kube-rbac-proxy/0.log" Jul 23 17:52:15.496666 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:15.496634 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4dzk_538386f2-bb08-4d4b-8445-84ca1f37b434/init-textfile/0.log" Jul 23 17:52:15.973525 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:15.973496 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-4jbv7_434dd052-9b1f-4780-98d5-6a9255044a71/prometheus-operator-admission-webhook/0.log" Jul 23 17:52:17.454656 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.454621 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-6rdms_a5a8698a-f0bf-43a4-bb97-5614ba1adc68/networking-console-plugin/0.log" Jul 23 17:52:17.462607 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.462562 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc"] Jul 23 17:52:17.463088 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463059 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" Jul 23 17:52:17.463088 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463079 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463104 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463109 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a38dbe-db53-47f6-a35d-ed2d85da9b4d" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463117 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463124 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463132 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.463218 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463139 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.463506 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463224 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d59910-55e1-4fb8-99ec-da17089a960f" containerName="istio-proxy" Jul 23 17:52:17.463506 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463238 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.463506 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.463248 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7844ed9-cfcf-483e-a470-8e7caf47a7c0" containerName="istio-proxy" Jul 23 17:52:17.467915 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.467886 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.470902 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.470873 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5mz8v\"/\"kube-root-ca.crt\"" Jul 23 17:52:17.472059 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.472038 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5mz8v\"/\"openshift-service-ca.crt\"" Jul 23 17:52:17.472217 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.472037 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5mz8v\"/\"default-dockercfg-htj2r\"" Jul 23 17:52:17.475539 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.475016 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc"] Jul 23 17:52:17.484010 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.483975 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-proc\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.484275 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.484254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-podres\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.484375 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.484331 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-lib-modules\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.484443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.484375 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-sys\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.484443 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.484418 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq65w\" (UniqueName: \"kubernetes.io/projected/532d791f-67ea-46f0-86e0-041972bf279a-kube-api-access-dq65w\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585568 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585527 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-proc\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585592 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-podres\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-lib-modules\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585649 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-sys\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq65w\" (UniqueName: \"kubernetes.io/projected/532d791f-67ea-46f0-86e0-041972bf279a-kube-api-access-dq65w\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585762 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-proc\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585942 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585762 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-sys\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585942 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-podres\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.585942 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.585791 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/532d791f-67ea-46f0-86e0-041972bf279a-lib-modules\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.595619 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.595585 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq65w\" (UniqueName: \"kubernetes.io/projected/532d791f-67ea-46f0-86e0-041972bf279a-kube-api-access-dq65w\") pod \"perf-node-gather-daemonset-pnrlc\" (UID: \"532d791f-67ea-46f0-86e0-041972bf279a\") " pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.780025 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.779927 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:17.933488 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:17.933455 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc"] Jul 23 17:52:17.935378 ip-10-0-136-29 kubenswrapper[2584]: W0723 17:52:17.935324 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod532d791f_67ea_46f0_86e0_041972bf279a.slice/crio-f33dfdf3999e499fd08814443e013af3f43e4a54302b67171f407e6bb2a1258e WatchSource:0}: Error finding container f33dfdf3999e499fd08814443e013af3f43e4a54302b67171f407e6bb2a1258e: Status 404 returned error can't find the container with id f33dfdf3999e499fd08814443e013af3f43e4a54302b67171f407e6bb2a1258e Jul 23 17:52:18.224719 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.224693 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/1.log" Jul 23 17:52:18.229808 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.229781 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kfj29_1ef33513-8b26-482f-84eb-20918d4d1b74/console-operator/2.log" Jul 23 17:52:18.613114 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.613017 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" event={"ID":"532d791f-67ea-46f0-86e0-041972bf279a","Type":"ContainerStarted","Data":"36d66aed5e2ddd1f210e1257fd4e9e1b8bf5afef4380de8385b1ad3e49f7b35b"} Jul 23 17:52:18.613114 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.613070 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" event={"ID":"532d791f-67ea-46f0-86e0-041972bf279a","Type":"ContainerStarted","Data":"f33dfdf3999e499fd08814443e013af3f43e4a54302b67171f407e6bb2a1258e"} Jul 23 17:52:18.613554 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.613123 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:18.631490 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.631433 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" podStartSLOduration=1.6314128449999998 podStartE2EDuration="1.631412845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:52:18.630094755 +0000 UTC m=+2859.983956134" watchObservedRunningTime="2026-07-23 17:52:18.631412845 +0000 UTC m=+2859.985274225" Jul 23 17:52:18.827310 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:18.827277 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-jkhnd_e2df6662-2c8d-47a9-9457-8a75d33d7579/download-server/0.log" Jul 23 17:52:19.457087 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:19.457060 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-gc7kd_8f904527-8175-4f4c-8a6d-cd33aad0fc65/volume-data-source-validator/0.log" Jul 23 17:52:20.689665 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:20.689637 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p8jvx_5f164089-c9c0-479a-aa51-c2def9952c25/dns/0.log" Jul 23 17:52:20.713274 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:20.713241 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p8jvx_5f164089-c9c0-479a-aa51-c2def9952c25/kube-rbac-proxy/0.log" Jul 23 17:52:20.789858 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:20.789830 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hzlv7_cc5cb2c4-fa6e-4c4a-a487-a0b01f5fc7cf/dns-node-resolver/0.log" Jul 23 17:52:21.547266 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:21.547236 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g7x68_497a9838-9e96-4f68-8e2a-db3b3172959e/node-ca/0.log" Jul 23 17:52:22.626663 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.626627 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sw65h_132969f0-927a-4c64-b0ba-321a5a51f30d/istio-proxy/0.log" Jul 23 17:52:22.768658 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.768626 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rglvp_1f8fb21c-fc35-40ae-8ee8-b74f4bae46da/discovery/0.log" Jul 23 17:52:22.790553 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.790529 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7466c75dcd-bl6q4_2c79f73f-844e-4588-94b9-77f13a8a2086/kube-auth-proxy/0.log" Jul 23 17:52:22.848023 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.847995 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/0.log" Jul 23 17:52:22.873936 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.873900 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/1.log" Jul 23 17:52:22.906925 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.906841 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-47hhw_1c4f0d2f-57ca-43f4-93c5-b2229161fde6/payload-pre-processing/0.log" Jul 23 17:52:22.963750 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:22.963723 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f9cf5b5b-kqk57_3021ee46-d3b1-4152-bec2-24e89cabe038/router/0.log" Jul 23 17:52:23.716659 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:23.716627 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzknq_3ce2a2be-034c-47b2-8a3d-3d38cda5b020/serve-healthcheck-canary/0.log" Jul 23 17:52:24.317498 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.317466 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-82hnk_5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb/insights-operator/0.log" Jul 23 17:52:24.318006 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.317988 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-82hnk_5f868cd3-fda3-41e9-aff8-ea7ce09fa0eb/insights-operator/1.log" Jul 23 17:52:24.338559 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.338533 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5gr9t_57824bfa-1b6c-460f-87f3-a86549527f17/kube-rbac-proxy/0.log" Jul 23 17:52:24.361129 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.361098 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5gr9t_57824bfa-1b6c-460f-87f3-a86549527f17/exporter/0.log" Jul 23 17:52:24.383352 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.383318 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5gr9t_57824bfa-1b6c-460f-87f3-a86549527f17/extractor/0.log" Jul 23 17:52:24.629139 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:24.629061 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5mz8v/perf-node-gather-daemonset-pnrlc" Jul 23 17:52:27.374349 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.374320 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-f96g6_180c167e-3fa3-4214-b2e5-74796644b1d5/server/0.log" Jul 23 17:52:27.633655 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.633602 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9b469455-qb96r_16087e24-40c7-44f8-8e2d-a683abb54b27/manager/0.log" Jul 23 17:52:27.659734 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.659710 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9b469455-qb96r_16087e24-40c7-44f8-8e2d-a683abb54b27/copy-manifests/0.log" Jul 23 17:52:27.828031 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.827935 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sw65h_132969f0-927a-4c64-b0ba-321a5a51f30d/istio-proxy/0.log" Jul 23 17:52:27.959653 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.959619 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rglvp_1f8fb21c-fc35-40ae-8ee8-b74f4bae46da/discovery/0.log" Jul 23 17:52:27.969850 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:27.969827 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7466c75dcd-bl6q4_2c79f73f-844e-4588-94b9-77f13a8a2086/kube-auth-proxy/0.log" Jul 23 17:52:28.005181 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:28.005148 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/0.log" Jul 23 17:52:28.032839 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:28.032801 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/1.log" Jul 23 17:52:28.054847 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:28.054798 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-47hhw_1c4f0d2f-57ca-43f4-93c5-b2229161fde6/payload-pre-processing/0.log" Jul 23 17:52:28.091014 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:28.090912 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f9cf5b5b-kqk57_3021ee46-d3b1-4152-bec2-24e89cabe038/router/0.log" Jul 23 17:52:29.382017 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.381971 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sw65h_132969f0-927a-4c64-b0ba-321a5a51f30d/istio-proxy/0.log" Jul 23 17:52:29.515414 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.515355 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-rglvp_1f8fb21c-fc35-40ae-8ee8-b74f4bae46da/discovery/0.log" Jul 23 17:52:29.530824 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.530739 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7466c75dcd-bl6q4_2c79f73f-844e-4588-94b9-77f13a8a2086/kube-auth-proxy/0.log" Jul 23 17:52:29.571646 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.571615 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/0.log" Jul 23 17:52:29.600257 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.600217 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rh6gv_a13b25b8-9ce2-4354-8629-a744a438f99e/istio-proxy/1.log" Jul 23 17:52:29.626465 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.626438 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-47hhw_1c4f0d2f-57ca-43f4-93c5-b2229161fde6/payload-pre-processing/0.log" Jul 23 17:52:29.675451 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:29.675416 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f9cf5b5b-kqk57_3021ee46-d3b1-4152-bec2-24e89cabe038/router/0.log" Jul 23 17:52:34.356034 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:34.356004 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-wk6sm_967a098b-fcef-4b09-9dd5-c7bb80868a61/migrator/0.log" Jul 23 17:52:34.383315 ip-10-0-136-29 kubenswrapper[2584]: I0723 17:52:34.383287 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-wk6sm_967a098b-fcef-4b09-9dd5-c7bb80868a61/graceful-termination/0.log"